This refactors the thin_archive tool to use emitters and scanners
to note that when linking to a thin archive, you must also depend on
the children of that thin archive. Failing to do so is an error,
because a changed .o does not necessarily lead to a different .a,
which would subvert the SCons dependency mechanism.
This also includes a refactoring of the ABILINK tool to use a similar
mechanism, to achieve the opposite effect. For ABILINK, we want to
depend not on the actual .so, but on the hash of its abidw result. We
use emitters, actions, and scanners to produce an associated .abidw
file for each .so we build, and then update the scanner to depend on
the .abidw of our libraries, not the library itself. This allows us to
elide needless relinks.
* The integration_tests alias now depends on the individual tests directly
rather than the whole directory.
* Both list files now just depend on the list of tests rather than the tests
themselves.
* The string printed during execution is now evaluated at the right time so
we don't need to print it separately.
* Fix installing tests to the build/unitests/ directory on windows.
From the python docs: "In general, the key and reverse conversion processes
are much faster than specifying an equivalent cmp function. This is because
cmp is called multiple times for each list element while key and reverse touch
each element only once."
When the compilation_db scons emitter saved compilation database
entried it always used the "global" environment which might not match
the compilation command line actually used to build the target.
That meant that using eg.:
env.Library("foo", ["foo.cpp"], CXXFLAGS=["-Dfoo"])
Would not write the extra "-Dfoo" flag actually used to build the foo
library to the compile_commands.json file.
Instead save the environment used to generate the target and use that
when writing the compile_command.json entry.
By default on SCons, the Library and StaticLibrary builders are
the same object, so adding the target_factory to StaticLibrary is
sufficient. If they aren't the same, then even more important
to only modify SaticLibrary, since Library may have been re-purposed
to mean something else. In our case it does mean something different
when using --link-model=dynamic, so this oversight inadvertently changed
the signature calculations for dynamic libaries.
Also includes a drive-by fix to add LoadableModule to the builders
affected by the abilink.py tool, since a LoadableModule is also
a shared library on the platforms where abidw is in play.
Split out the passthrough tests into separate suites. The MongoDB
deployment is started up by resmoke.py so that we can record the
success/failure of each individual test in MCI.
Added support for parallel execution of tests by dispatching to
multiple MongoDB deployments.
Added support for grouping different kinds of tests (e.g. C++ unit
tests, dbtests, and jstests) so that they can be run together. This
allows for customizability in specifying what tests to execute when
changes are made to a particular part of the code.
Includes:
- Smoke.py options for filtering by jstest tags
- New resmoke.py test utility in v0 alpha
- Lots of example resmoke configurations
- SCons integration for unittests
- Sample tagged jstests
Includes:
- Smoke.py options for filtering by jstest tags
- New resmoke.py test utility in v0 alpha
- Lots of example resmoke configurations
- SCons integration for unittests
- Sample tagged jstests
1. VCXProj now builds all .cpp files itself
2. Moved obj files to Build/ to keep repo clean
3. Split jstoh so that it can be called by vcxproj file
4. Make_Vcxproj - fixed to handle cl temp files, fixes now that vcxproj builds
generated files, and fix _TARGET_ search and replace
Note: Depends on python installation
To achieve this, the "direct prerequisites" for a File node are stored in the
node's attributes object, instead of in the build environment for the node.
This allows the emitter for nodes to amend _other nodes_ prerequisites. After that,
this change is trivial. Introduce a LIBDEPS_DEPENDENTS environment variable that sets
the named dependents of a node, while the existing LIBDEPS variable may be considered
the direct prerequisites.
These tests exist to facilitate writing integration tests for MongoDB modules,
and could also be used to register integration tests for core MongoDB components
that are not written as jstests or dbtests.
The previous implementation was not recursively following the LIBDEPS hierarchy, looking for SYSLIBDEPS
to add. Instead, it was trying to walk a hierachy like LIBDEPS rooted in the SYSLIBDEPS variable. This
isn't really a meaningful behavior, since SYSLIBDEPS always list system libraries, which are always leaf
nodes in the dependency graph for our build system.
SCons (correctly) considers two link command lines different if the order of
object files or static libraries changes. As a result, the libdeps system needs
to produce consistent ordering of these files in the $_LIBDEPS expansion. This
patch achieves this by sorting the _LIBDEPS candidate expansion by the string
name of the expanded objects.
This problem is akin to that from SERVER-5254, which was solved in a similar
manner.
Tests are registered with env.RegisterUnitTest(), or by compiling a C++ unit
test with
env.CppUnitTest('test_name', [test_source_list], LIBDEPS=[...])
The result is that SCons knows how to build a file "build/unittests.txt", one line per test.
Turns out that SCons.Node objects are compared on object identity (pointer),
not some value that's stable across invocations of SCons. We now sort by
the "name" of the node (its string representation), instead, which stops
spurious builds.
This is used to fix linking of programs on Windows that depend on static
initializer execution in object files that don't contain symbols referenced
directly by the program at compile time (i.e., Command registration).
It also unifies the compilation of these sorts of programs on Windows and POSIX
systems, eliminating the use of a MergeLibrary when linking mongod and mongos.
This extension to libdeps could also be used to reimagine MergeLibrary to have
an implementation shared between Posix and Windows systems. The existing
MergeLibrary system uses relocatable objects on Posix systems, and thus could
behave differently from the one on Windows, with respect to the exeuction of
static initializers. That change is recommended future work.
Newer versions of SCons are more consistent about making lists out of TARGET and
SOURCE fields than older (1.2.0 and older) versions. This patch bandaids that
problem, which was exposed by the libdeps module.
This patch is a reorganization of our build files, which brings them slightly
closer in line with standard SCons organization.
In particular, the SConstruct file sets up the various "build environment"
objects, by examining the local system and command line parameters. Then, it
delegates to some SConscript files, which describe build rules, like how to
compile "mongod" from source.
Typically, you would create several SConscript files for a project this large,
after breaking the project into logical sub projects, such as "platform
abstraction", "data manager", "query optimizer", etc. That will be future work.
For now, we only separate out the special rules for executing smoke tests into
SConscript.smoke. Pretty much all other build rules are in src/mongo/SConscript.
"tools" are placed in site_scons/site_tools.
This patch also includes better support for building and tracking dependencies
among static libraries ("libdeps" and "MergeLibrary"), and some incumbent, minor
restructuring.
This patch introduces a "warning" message from SCons about framework.o having
two rules that generate it. It is harmless, for now, and will be removed in
future work.
Future work also includes eliminating use of the SCons "Glob" utility, and
restructuring the source code into sensible components.
