2017-11-10 19:40:55 +01:00
|
|
|
|
# Notes about the `tools/icu` subdirectory
|
|
|
|
|
|
2018-04-14 15:06:43 +02:00
|
|
|
|
This directory contains tools, data, and information about the [ICU](http://icu-project.org)
|
2018-02-12 08:31:55 +01:00
|
|
|
|
(International Components for Unicode) integration. ICU is used to provide
|
|
|
|
|
internationalization functionality.
|
|
|
|
|
|
|
|
|
|
- `patches/` are one-off patches, actually entire source file replacements,
|
|
|
|
|
organized by ICU version number.
|
|
|
|
|
- `icu_small.json` controls the "small" (English only) ICU. It is input to
|
|
|
|
|
`icutrim.py`
|
|
|
|
|
- `icu-generic.gyp` is the build file used for most ICU builds within ICU.
|
|
|
|
|
<!-- have fun -->
|
|
|
|
|
- `icu-system.gyp` is an alternate build file used when `--with-intl=system-icu`
|
|
|
|
|
is invoked. It builds against the `pkg-config` located ICU.
|
|
|
|
|
- `iculslocs.cc` is source for the `iculslocs` utility, invoked by `icutrim.py`
|
|
|
|
|
as part of repackaging. Not used separately. See source for more details.
|
2017-11-10 19:40:55 +01:00
|
|
|
|
- `no-op.cc` — empty function to convince gyp to use a C++ compiler.
|
|
|
|
|
- `README.md` — you are here
|
|
|
|
|
- `shrink-icu-src.py` — this is used during upgrade (see guide below)
|
build, i18n: improve Intl build, add "--with-intl"
The two main goals of this change are:
- To make it easier to build the Intl option using ICU (particularly,
using a newer ICU than v8/Chromium's version)
- To enable a much smaller ICU build with only English support The goal
here is to get node.js binaries built this way by default so that the
Intl API can be used. Additional data can be added at execution time
(see Readme and wiki)
More details are at https://github.com/joyent/node/pull/7719
In particular, this change adds the "--with-intl=" configure option to
provide more ways of building "Intl":
- "full-icu" picks up an ICU from deps/icu
- "small-icu" is similar, but builds only English
- "system-icu" uses pkg-config to find an installed ICU
- "none" does nothing (no Intl)
For Windows builds, the "full-icu" or "small-icu" options are added to
vcbuild.bat.
Note that the existing "--with-icu-path" option is not removed from
configure, but may not be used alongside the new option.
Wiki changes have already been made on
https://github.com/joyent/node/wiki/Installation
and a new page created at
https://github.com/joyent/node/wiki/Intl
(marked as provisional until this change lands.)
Summary of changes:
* README.md : doc updates
* .gitignore : added "deps/icu" as this is the location where ICU is
unpacked to.
* Makefile : added the tools/icu/* files to cpplint, but excluded a
problematic file.
* configure : added the "--with-intl" option mentioned above.
Calculate at config time the list of ICU source files to use and data
packaging options.
* node.gyp : add the new files src/node_i18n.cc/.h as well as ICU
linkage.
* src/node.cc : add call into
node::i18n::InitializeICUDirectory(icu_data_dir) as well as new
--icu-data-dir option and NODE_ICU_DATA env variable to configure ICU
data loading. This loading is only relevant in the "small"
configuration.
* src/node_i18n.cc : new source file for the above Initialize..
function, to setup ICU as needed.
* tools/icu : new directory with some tools needed for this build.
* tools/icu/icu-generic.gyp : new .gyp file that builds ICU in some new
ways, both on unix/mac and windows.
* tools/icu/icu-system.gyp : new .gyp file to build node against a
pkg-config detected ICU.
* tools/icu/icu_small.json : new config file for the "English-only" small
build.
* tools/icu/icutrim.py : new tool for trimming down ICU data. Reads the
above .json file.
* tools/icu/iculslocs.cc : new tool for repairing ICU data manifests
after trim operation.
* tools/icu/no-op.cc : dummy file to force .gyp into using a C++ linker.
* vcbuild.bat : added small-icu and full-icu options, to call into
configure.
* Fixed toolset dependencies, see
https://github.com/joyent/node/pull/7719#issuecomment-54641687
Note that because of a bug in gyp {CC,CXX}_host must also be set.
Otherwise gcc/g++ will be used by default for part of the build.
Reviewed-by: Trevor Norris <trev.norris@gmail.com>
Reviewed-by: Fedor Indutny <fedor@indutny.com>
2014-09-05 07:03:24 +02:00
|
|
|
|
|
2016-08-06 12:58:55 +02:00
|
|
|
|
## How to upgrade ICU
|
2016-04-09 04:03:24 +02:00
|
|
|
|
|
2018-03-04 14:46:49 +01:00
|
|
|
|
- Make sure your Node.js workspace is clean (clean `git status`) should be
|
2018-02-12 08:31:55 +01:00
|
|
|
|
sufficient.
|
2018-03-04 14:46:49 +01:00
|
|
|
|
- Configure Node.js with the specific [ICU version](http://icu-project.org/download)
|
2018-02-12 08:31:55 +01:00
|
|
|
|
you want to upgrade to, for example:
|
2016-04-09 04:03:24 +02:00
|
|
|
|
|
2016-08-06 12:58:55 +02:00
|
|
|
|
```shell
|
2016-04-09 04:03:24 +02:00
|
|
|
|
./configure \
|
|
|
|
|
--with-intl=small-icu \
|
2016-10-20 01:53:39 +02:00
|
|
|
|
--with-icu-source=http://download.icu-project.org/files/icu4c/58.1/icu4c-58_1-src.tgz
|
2016-04-09 04:03:24 +02:00
|
|
|
|
make
|
|
|
|
|
```
|
|
|
|
|
|
2017-11-10 19:40:55 +01:00
|
|
|
|
> _Note_ in theory, the equivalent `vcbuild.bat` commands should work also,
|
2018-01-30 02:08:21 +01:00
|
|
|
|
> but the commands below are makefile-centric.
|
2016-04-09 04:03:24 +02:00
|
|
|
|
|
2018-02-12 08:31:55 +01:00
|
|
|
|
- If there are ICU version-specific changes needed, you may need to make changes
|
|
|
|
|
in `icu-generic.gyp` or add patch files to `tools/icu/patches`.
|
|
|
|
|
- Specifically, look for the lists in `sources!` in the `icu-generic.gyp` for
|
|
|
|
|
files to exclude.
|
2016-04-09 04:03:24 +02:00
|
|
|
|
|
2018-03-04 14:46:49 +01:00
|
|
|
|
- Verify the Node.js build works:
|
2016-04-09 04:03:24 +02:00
|
|
|
|
|
2016-08-06 12:58:55 +02:00
|
|
|
|
```shell
|
2016-04-09 04:03:24 +02:00
|
|
|
|
make test-ci
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Also running
|
2017-04-21 21:55:51 +02:00
|
|
|
|
|
|
|
|
|
<!-- eslint-disable strict -->
|
2017-11-10 19:40:55 +01:00
|
|
|
|
|
2016-04-09 04:03:24 +02:00
|
|
|
|
```js
|
2017-07-03 02:05:59 +02:00
|
|
|
|
new Intl.DateTimeFormat('es', {month: 'long'}).format(new Date(9E8));
|
2016-04-09 04:03:24 +02:00
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
…Should return `January` not `enero`.
|
|
|
|
|
|
|
|
|
|
- Now, copy `deps/icu` over to `deps/icu-small`
|
|
|
|
|
|
2016-08-06 12:58:55 +02:00
|
|
|
|
```shell
|
2016-04-09 04:03:24 +02:00
|
|
|
|
python tools/icu/shrink-icu-src.py
|
|
|
|
|
```
|
|
|
|
|
|
2018-03-04 14:46:49 +01:00
|
|
|
|
- Now, do a clean rebuild of Node.js to test:
|
2016-04-09 04:03:24 +02:00
|
|
|
|
|
2016-08-06 12:58:55 +02:00
|
|
|
|
```shell
|
2017-11-10 19:40:55 +01:00
|
|
|
|
make -k distclean
|
|
|
|
|
./configure
|
2016-04-09 04:03:24 +02:00
|
|
|
|
make
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
- Test this newly default-generated Node.js
|
2017-04-21 21:55:51 +02:00
|
|
|
|
|
|
|
|
|
<!-- eslint-disable strict -->
|
2017-11-10 19:40:55 +01:00
|
|
|
|
|
2016-04-09 04:03:24 +02:00
|
|
|
|
```js
|
|
|
|
|
process.versions.icu;
|
2017-04-21 21:55:51 +02:00
|
|
|
|
new Intl.DateTimeFormat('es', {month: 'long'}).format(new Date(9E8));
|
2016-04-09 04:03:24 +02:00
|
|
|
|
```
|
|
|
|
|
|
2017-11-10 19:40:55 +01:00
|
|
|
|
(This should print your updated ICU version number, and also `January` again.)
|
2016-04-09 04:03:24 +02:00
|
|
|
|
|
2017-11-10 19:40:55 +01:00
|
|
|
|
You are ready to check in the updated `deps/small-icu`. This is a big commit,
|
|
|
|
|
so make this a separate commit from the smaller changes.
|
2016-04-09 04:03:24 +02:00
|
|
|
|
|
2018-03-04 14:46:49 +01:00
|
|
|
|
- Now, rebuild the Node.js license.
|
2016-10-20 01:53:39 +02:00
|
|
|
|
|
|
|
|
|
```shell
|
|
|
|
|
# clean up - remove deps/icu
|
|
|
|
|
make clean
|
|
|
|
|
tools/license-builder.sh
|
|
|
|
|
```
|
|
|
|
|
|
2018-10-04 22:44:34 +02:00
|
|
|
|
- Update the URL and hash for the full ICU file in `tools/icu/current_ver.dep`.
|
|
|
|
|
It should match the ICU URL used in the first step. When this is done, the
|
|
|
|
|
following should build with full ICU.
|
2016-04-09 04:03:24 +02:00
|
|
|
|
|
2016-08-06 12:58:55 +02:00
|
|
|
|
```shell
|
2016-04-09 04:03:24 +02:00
|
|
|
|
# clean up
|
|
|
|
|
rm -rf out deps/icu deps/icu4c*
|
|
|
|
|
./configure --with-intl=full-icu --download=all
|
|
|
|
|
make
|
|
|
|
|
make test-ci
|
|
|
|
|
```
|
|
|
|
|
|
2018-10-04 22:44:34 +02:00
|
|
|
|
- commit the change to `tools/icu/current_ver.dep` and `LICENSE` files.
|
2016-04-09 04:03:24 +02:00
|
|
|
|
|
2018-02-12 08:31:55 +01:00
|
|
|
|
- Note: To simplify review, I often will “pre-land” this patch, meaning that
|
|
|
|
|
I run the patch through `curl -L https://github.com/nodejs/node/pull/xxx.patch
|
|
|
|
|
| git am -3 --whitespace=fix` per the collaborator’s guide… and then push that
|
|
|
|
|
patched branch into my PR's branch. This reduces the whitespace changes that
|
|
|
|
|
show up in the PR, since the final land will eliminate those anyway.
|
2017-11-10 19:40:55 +01:00
|
|
|
|
|
2016-04-09 04:03:24 +02:00
|
|
|
|
-----
|
|
|
|
|
|
2017-11-10 19:40:55 +01:00
|
|
|
|
## Postscript about the tools
|
2016-04-09 04:03:24 +02:00
|
|
|
|
|
2018-03-04 14:46:49 +01:00
|
|
|
|
The files in this directory were written for the Node.js effort.
|
2017-11-10 19:40:55 +01:00
|
|
|
|
It was the intent of their author (Steven R. Loomis / srl295) to
|
|
|
|
|
merge them upstream into ICU, pending much discussion within the
|
|
|
|
|
ICU-TC.
|
build, i18n: improve Intl build, add "--with-intl"
The two main goals of this change are:
- To make it easier to build the Intl option using ICU (particularly,
using a newer ICU than v8/Chromium's version)
- To enable a much smaller ICU build with only English support The goal
here is to get node.js binaries built this way by default so that the
Intl API can be used. Additional data can be added at execution time
(see Readme and wiki)
More details are at https://github.com/joyent/node/pull/7719
In particular, this change adds the "--with-intl=" configure option to
provide more ways of building "Intl":
- "full-icu" picks up an ICU from deps/icu
- "small-icu" is similar, but builds only English
- "system-icu" uses pkg-config to find an installed ICU
- "none" does nothing (no Intl)
For Windows builds, the "full-icu" or "small-icu" options are added to
vcbuild.bat.
Note that the existing "--with-icu-path" option is not removed from
configure, but may not be used alongside the new option.
Wiki changes have already been made on
https://github.com/joyent/node/wiki/Installation
and a new page created at
https://github.com/joyent/node/wiki/Intl
(marked as provisional until this change lands.)
Summary of changes:
* README.md : doc updates
* .gitignore : added "deps/icu" as this is the location where ICU is
unpacked to.
* Makefile : added the tools/icu/* files to cpplint, but excluded a
problematic file.
* configure : added the "--with-intl" option mentioned above.
Calculate at config time the list of ICU source files to use and data
packaging options.
* node.gyp : add the new files src/node_i18n.cc/.h as well as ICU
linkage.
* src/node.cc : add call into
node::i18n::InitializeICUDirectory(icu_data_dir) as well as new
--icu-data-dir option and NODE_ICU_DATA env variable to configure ICU
data loading. This loading is only relevant in the "small"
configuration.
* src/node_i18n.cc : new source file for the above Initialize..
function, to setup ICU as needed.
* tools/icu : new directory with some tools needed for this build.
* tools/icu/icu-generic.gyp : new .gyp file that builds ICU in some new
ways, both on unix/mac and windows.
* tools/icu/icu-system.gyp : new .gyp file to build node against a
pkg-config detected ICU.
* tools/icu/icu_small.json : new config file for the "English-only" small
build.
* tools/icu/icutrim.py : new tool for trimming down ICU data. Reads the
above .json file.
* tools/icu/iculslocs.cc : new tool for repairing ICU data manifests
after trim operation.
* tools/icu/no-op.cc : dummy file to force .gyp into using a C++ linker.
* vcbuild.bat : added small-icu and full-icu options, to call into
configure.
* Fixed toolset dependencies, see
https://github.com/joyent/node/pull/7719#issuecomment-54641687
Note that because of a bug in gyp {CC,CXX}_host must also be set.
Otherwise gcc/g++ will be used by default for part of the build.
Reviewed-by: Trevor Norris <trev.norris@gmail.com>
Reviewed-by: Fedor Indutny <fedor@indutny.com>
2014-09-05 07:03:24 +02:00
|
|
|
|
|
|
|
|
|
`icu_small.json` is somewhat node-specific as it specifies a "small ICU"
|
|
|
|
|
configuration file for the `icutrim.py` script. `icutrim.py` and
|
|
|
|
|
`iculslocs.cpp` may themselves be superseded by components built into
|
2017-11-10 19:40:55 +01:00
|
|
|
|
ICU in the future. As of this writing, however, the tools are separate
|
2018-03-04 14:46:49 +01:00
|
|
|
|
entities within Node.js, although theyare being scrutinized by interested
|
2017-11-10 19:40:55 +01:00
|
|
|
|
members of the ICU-TC. The “upstream” ICU bugs are given below.
|
build, i18n: improve Intl build, add "--with-intl"
The two main goals of this change are:
- To make it easier to build the Intl option using ICU (particularly,
using a newer ICU than v8/Chromium's version)
- To enable a much smaller ICU build with only English support The goal
here is to get node.js binaries built this way by default so that the
Intl API can be used. Additional data can be added at execution time
(see Readme and wiki)
More details are at https://github.com/joyent/node/pull/7719
In particular, this change adds the "--with-intl=" configure option to
provide more ways of building "Intl":
- "full-icu" picks up an ICU from deps/icu
- "small-icu" is similar, but builds only English
- "system-icu" uses pkg-config to find an installed ICU
- "none" does nothing (no Intl)
For Windows builds, the "full-icu" or "small-icu" options are added to
vcbuild.bat.
Note that the existing "--with-icu-path" option is not removed from
configure, but may not be used alongside the new option.
Wiki changes have already been made on
https://github.com/joyent/node/wiki/Installation
and a new page created at
https://github.com/joyent/node/wiki/Intl
(marked as provisional until this change lands.)
Summary of changes:
* README.md : doc updates
* .gitignore : added "deps/icu" as this is the location where ICU is
unpacked to.
* Makefile : added the tools/icu/* files to cpplint, but excluded a
problematic file.
* configure : added the "--with-intl" option mentioned above.
Calculate at config time the list of ICU source files to use and data
packaging options.
* node.gyp : add the new files src/node_i18n.cc/.h as well as ICU
linkage.
* src/node.cc : add call into
node::i18n::InitializeICUDirectory(icu_data_dir) as well as new
--icu-data-dir option and NODE_ICU_DATA env variable to configure ICU
data loading. This loading is only relevant in the "small"
configuration.
* src/node_i18n.cc : new source file for the above Initialize..
function, to setup ICU as needed.
* tools/icu : new directory with some tools needed for this build.
* tools/icu/icu-generic.gyp : new .gyp file that builds ICU in some new
ways, both on unix/mac and windows.
* tools/icu/icu-system.gyp : new .gyp file to build node against a
pkg-config detected ICU.
* tools/icu/icu_small.json : new config file for the "English-only" small
build.
* tools/icu/icutrim.py : new tool for trimming down ICU data. Reads the
above .json file.
* tools/icu/iculslocs.cc : new tool for repairing ICU data manifests
after trim operation.
* tools/icu/no-op.cc : dummy file to force .gyp into using a C++ linker.
* vcbuild.bat : added small-icu and full-icu options, to call into
configure.
* Fixed toolset dependencies, see
https://github.com/joyent/node/pull/7719#issuecomment-54641687
Note that because of a bug in gyp {CC,CXX}_host must also be set.
Otherwise gcc/g++ will be used by default for part of the build.
Reviewed-by: Trevor Norris <trev.norris@gmail.com>
Reviewed-by: Fedor Indutny <fedor@indutny.com>
2014-09-05 07:03:24 +02:00
|
|
|
|
|
|
|
|
|
* [#10919](http://bugs.icu-project.org/trac/ticket/10919)
|
|
|
|
|
(experimental branch - may copy any source patches here)
|
|
|
|
|
* [#10922](http://bugs.icu-project.org/trac/ticket/10922)
|
|
|
|
|
(data packaging improvements)
|
|
|
|
|
* [#10923](http://bugs.icu-project.org/trac/ticket/10923)
|
|
|
|
|
(rewrite data building in python)
|