From d5a3db50c534b4e28461a519ee5dbaa71c68b48e Mon Sep 17 00:00:00 2001 From: Roland Kaufmann Date: Mon, 9 Dec 2013 11:44:38 +0100 Subject: [PATCH 1/2] Document modules and structures in the build system Give a brief description of each of the modules that comprises the build system, and the suffices that is used to form a virtual structure of variables for each project. --- cmake/OPM-CMake.md | 748 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 748 insertions(+) create mode 100644 cmake/OPM-CMake.md diff --git a/cmake/OPM-CMake.md b/cmake/OPM-CMake.md new file mode 100644 index 00000000..06a1183f --- /dev/null +++ b/cmake/OPM-CMake.md @@ -0,0 +1,748 @@ +OPM Build System +================ + +This is the documentation for the build system used in various OPM modules. +In the following, `xxx` is used as a placeholder for the project name (e.g. +"core"). + +Unlike traditional CMake files which is highly imperative, OPM projects +sets up declarative lists of prerequisites and content and rely on convention +and pre-made modules to do the build. Its goal is to replace but be +compatible with the old autotools-based system. + +## Notation + +In the build system to following abstract locations are referred to: + + + + + +
LocationDescription
Source tree + +This is where the source files are located. Usually this directory is created +by a `git clone`. You edit files in this directory. The build system on the +other hand will never touch these files; they could be read-only for that +matter. It should be located on a disk that is backed up. The source trees +for various OPM modules should be siblings to eachother. + +
Build tree + +This is where you do `make` (or `ninja`), and the compiler/linker will put +its output here. It may be the same directory as the source tree (which is +then called an "in-tree build"). However, it is recommended that you keep +it separate. Do `make clean && make distclean` to remove all files that +are created by the build (if you put it in the source directory). +You don't need to backup these files (since they can be generated from the +source); instead this directory should be located somewhere with fast +write access. The build trees for various OPM modules should be siblings +(unless they are subdirectories in their source trees). + +
Installation tree + +This is where the build system will put all the final libraries and headers +when running `make install`. +You can specify another location for the installation tree by setting the +CMake variable `CMAKE_INSTALL_PREFIX` on the command line (or use `--prefix=` +with the configure script). Notice that the portion of this path which +will become the new filesystem root should be specified with the environment +variable `DESTDIR`. + +
+ +## Use Cases + +This section lists some common use cases for adding new code to a project +with respect to the build system, and lists the steps that must be undertaken +to do the necessary modifications. + +### Adding a Translation Unit + +1. Put the file in a sub-directory of `opm/xxx`. + +2. Add the file name to the `MAIN_SOURCE_FILES` list in `CMakeLists_files.txt`. + Please keep this list sorted. + +3. If you are adding new interfaces that will be used by client code, add the + header to the `PUBLIC_HEADER_FILES`. Note that any `_impl` headers containing + template implementations must also be included. + +### Adding a Prerequisite + +1. Add the name of the prerequisite module to the `opm-xxx_DEPS` list in the file + `cmake/Modules/opm-xxx-prereqs.cmake`, where xxx is a placeholder for the module + name of your CMake project. + +2. If you need any CMake configuration defines available in your public _headers_, + add these to the `opm-xxx_CONFIG_VAR` list in the same file. Please refrain + from this practice as it imposes a requirement on the clients of your code to + gather the same configuration information and make it available in the units + which uses your headers. + +### Adding a Unit Test + +1. Put the source code in a single translation unit in directory `tests`. The + name of this unit should start with `test_`. + +2. Put any datafiles this code rely on in the same directory. The code should + assume that such datafiles are available in the current directory the program + is running from. The code should not write to these files, but rather make + a copy if it needs to modify them. + +3. Add the file name to the `TEST_SOURCE_FILES` list in `CMakeLists_files.txt`. + +4. Add the datafiles to the `TEST_DATA_FILES` list in the same files. The + files will be copied from the source tree into the target tree. + +### Adding a New Utility Program + +1. Put the source code of the utility in the `examples` directory. + +2. Add the name of the translation unit to the `PROGRAM_SOURCE_FILES` list + in `CMakeLists_files.txt`. + +### Creating a New Module + +1. Copy the directory `cmake/` and all sub-directories from opm-core. This + directory contains the common build system, and should ideally be identical + across OPM projects. Also copy the file `configure` in the project root. + +2. Create project-specific files using those from another project as a template. + The list of project specific files is in the section + [Modules Reference](#project-specific-files) below. + +3. Create a new file `cmake/Modules/opm-xxx-prereqs.cmake`, using one of the + existing ones as a template. + +4. Optionally, create a new file `cmake/Modules/Findopm-xxx.cmake`, using one + of the existing ones as a template. + +## Options + +These options regulate the behaviour of the build system. In addition to these +options, you can also set standard CMake options, or options for the +prerequisites, which is not documented here. If you run the configure script +with the `--help` option, it will print a text of what the options are called +when using the autotools-compatible wrapper. + + + + + + + + + + + + + + + +
OptionDescription
BUILD_EXAMPLES + +Include the examples when doing a build. Whenever you change something +in the library, however small, all the examples will also be rebuilt. +Default is ON. + +
BUILD_TESTING + +Include the unit tests when doing a build. Whenever you change something +in the library, however small, all the unit tests will also be rebuilt. +Default is ON. + +
PRECOMPILE_HEADERS + +Precompile common headers into a binary blob which is loaded on further +compilations. If your compiler supports this, it usually reduces build +time. It does not affect the running time of the code. Default is OFF. + +
SIBLING_SEARCH + +Search for OPM/DUNE prerequisites in sibling directories of the build +tree. Default is ON. + +
SUITESPARSE_USE_STATIC + +Link SuiteSparse/UMFPack statically. Using this option will enable you +to build an executable which has no external dependencies. The default +is to use shared libraries if those are available. + +
SYSTEM_DEBUG + +Put debug symbols in the system debug directory (`/usr/lib/debug`) as +this seems to be the only place which is searched by default by GDB. +Default is OFF, as it requires that you have write access to that +directory. Note that if you are doing a system installation (set +CMAKE_INSTALL_PREFIX=/usr), then the libraries will be put in this +location irregardless of this option. + +
USE_MPI + +Enable the code to use MPI for parallelization. Default is OFF. +Note: It is important that OPM and DUNE modules is either all +compiled with MPI support or that none is. The build system will +attempt to recognize inconsistencies. + +
USE_OPENMP + +Enable the code to use OpenMP for parallelization. Default is ON. + +
USE_RUNPATH + +Remember the directories from where the prerequisites were found +when building. Default is ON, which enables you to run without +setting PATH all over the place in your build directories. When +creating an installation package, this should be set off. + +
USE_UNDERSCORING + +Assume that Fortran externals have an underscore suffix instead +of checking this with an actual compiler. If you set this option, +you can use Fortran libraries (notably BLAS and LAPACK) without +having a Fortran compiler installed. The default is OFF. + +
USE_VERSIONED_DIR + +Put libraries in a directory which includes the label of the project, +e.g. `/usr/lib/opm-core/2013.10`. Default is OFF. + +
WITH_NATIVE + +Optimize for the instruction set of the build machine. This is +a good idea if you are building the library on the same machine +as you will be using the library. Default is ON. + +
WHOLE_PROG_OPTIM + +Perform an extra round of optimization when linking the program. +(Usually the compiler only optimizes within the translation unit). +If your compiler supports this, it usually leads to a faster runtime. +Default is OFF. + +
+ +## Modules Reference + +### Project-specific Files + +All of these files are in the project root, except for `opm-xxx.m4` +which is in the `m4` directory. (`dunecontrol` always adds this +subdirectory for all the prerequisites listed in `dune.module`, to the +search path). + + + + + + + +
FileDescription
CMakeLists.txt + +Project-specific customizations to the build, such as filtering out source +files based on the availability of prerequisites, or adding configuration +variables only the implementation depends on. +Prefer to do customizations in the hooks available to this file rather than +adding ad-hoc code to the build system itself, to keep the `cmake/` directory +unified across projects. + +
CMakeLists_files.txt + +List of all compilation modules in the project, test cases and public +headers that should be installed in the distribution package. The contents +of these lists are distributed to project-specific variables by the build +system. + +
CTestConfig.cmake + +Settings for submitting result of tests to CDash. The default is setup +to submit to [the official CDash panel](http://www.opm-project.org/CDash/) +and does not need to be changed if your module has a panel there. + +
dune.module + +Information about the project such as name, release label, link version +and maintainer. Also specify dependencies to other OPM/DUNE-projects so +that dunecontrol can build in correct order. (Note that the full list of +dependencies is taken from opm-xxx-prereqs.cmake and not from here). +Since this file must be present before the build starts (for dunecontrol), +the version information is kept here. + +
opm_xxx.m4 + +Tell the generic opm.m4 module which name it should request pkg-config +for. This module is used by autotools-projects which link to OPM. +Notice that dashes is replaced by underscore in the filename to be +compatible with M4. (The actual name of the file doesn't matter to the +autotools build system). The contents of this file is mostly boiler-plate +where the names need to be changed to the project in question. (The +DUNE build system assumes the presence of macros based on the project +name). + +
+ +### Project Modules + +These modules contains the dependency information for this project, so +the build system can set up the prerequisite list correctly and probe +for other modules automatically. (This replaces hard-coded calls to +find_library in the CMakeLists.txt file). + + + + +
File (.cmake)Description
xxx-prereqs + +List prerequisite modules and defines used in public headers. Each module +must have such a "declaration", and this must be made available to every +other projects as well (which is why this is located in `cmake/Modules`). + +
Findxxx + +CMake modules which locates module `xxx` in the system directories. As +the `opm-xxx-config.cmake` is made available together with the libraries +and headers, these modules are not really needed (for OPM modules). + +
+ +### Generated Files + +These files are generated by the build system and exists in the _build_ tree, +not in the source tree. They are documented here to make developers aware of +their role in the build system. + + + + + + + + +
FileDescription
config.h + +Settings variables which the build system has configured and make available +to the source code. This file is **not** installed amongst the headers, so +you should never include this in a public header, even if you need the value +in one of these variables. Instead, you must rely on the client code to +configure this variable for you. + +
opm-xxx.pc + +pkg-config information file to locate the **build** tree. This is used by +the autotools build files, but can also be handy when manually building +small test programs for which you don't generate an own build system. + +
opm-xxx-config.cmake + +CMake information file to locate the **build** tree. This file is imported +when this project is set up as a prerequisite. + +
opm-xxx-install.pc + +pkg-config information file to locate the **installation** tree. It is +the same as `opm-xxx.pc` except that the paths are switched. When the +project is installed, this file is installed instead (under `lib/pkgconfig` +relative to the installation root). This directory should hence be put +in the search path to pkg-config to use the installed package. Before +installation, this file is worthless and should not be included, because +it does not refer to the build tree at all. (Be careful not to mix +the build and the installation tree). +Notice that the build system will forward a bunch of public definitions +which should be available to compile code referring to this library. + +
opm-xxx-install.cmake + +CMake information file to locate the **installation** tree. It is the +same as `opm-xxx-config.cmake` except that the paths are switched. When +the project is installed, this file is installed instead (under `share/cmake` +relative to the installation root). + +
opm-xxx-config-version.cmake + +CMake wants to include this into the build _before_ it is determined whether +the library was found successfully (depending on the version number perhaps), +so this information is put in its own file. Since it is the same for the +build tree and the installation tree, it is shared in both those locations. + +
+ +### Utility Modules + +These modules consists of useful routines which is not OPM-specific and +that could be used in any projects. They don't depend on any other parts +of the build system. + + + + + + +
File (.cmake)Description
AddOptions + +Functions to add options to compiler command line (e.g. 'CXXFLAGS'). +This macro can add options to more than one language and/or configuration +at a time, and also automatically removes duplicates. + +
ConfigVars + +Functions to write values of selected variables to `config.h`. The +advantage of using this compared to a template file, is that other +modules can add their own variables to be written (a project doesn't +need to know which variables a prerequisite wants to have in config.h). + +
DuneCompat + +Modify `Makefile` so dunecontrol can infer source directory from it. +dunecontrol infers the source location of prerequisites from textual +analysis of the Makefile in their build tree. (dunecontrol cannot build +with Ninja anyway, so that is not a problem). + +
Duplicates + +Functions to remove duplicate values from a list of libraries, which +must always be done from the beginning in order to link correctly. + +
LibtoolArchives + +Write .la file which will make libtool find our library. This enables +users of our library to use libtool even if we did not do so ourselves. + +
+ +### Build System Modules + +These are the modules which comprises the OPM-specific parts of the +build system. The overall flow through the stages of the build is best +captured by reading through the `OpmLibMain.cmake` module. + + + + + + + + + + + + + + + +
File (.cmake)Description
configure + +Wrapper script which emulates an autotools front-end, making the build +system usable with dunecontrol. There is one in the project root directory +which just forwards everything to the main script in `cmake/Scripts`. + +
OpmCompile + +Set up a compilation target for the library itself. It is assumed that +each OPM project build exactly one library file containing all its code. +The files and compiler options are passed through the project variables +(see the section [Variables Reference](#variables-reference) below). + +
OpmDefaults + +If optimization and debugging settings are not given on the command line, +supply a set of reasonable defaults for the detected platform and +compiler. + +
OpmDistClean + +Add a target (`make distclean`) to the build system which can remove the +build files themselves from the build directory tree. + +
OpmDoc + +Add target for building documentation, primarily Doxygen class reference +from the code. + +
OpmFiles + +Load list of files from `CMakeLists_files.txt` and put into the applicable +variables. + +
OpmGrid + +Adds the grid type selection code to config.h which is needed by dune-grid +if you want to set up a default grid. This is currently not needed by any +OPM project, and is provided only for porting client projects which previously +used this functionality from the autotools version. + +
OpmInit + +Read the version information and project name from `dune.module`. + +
OpmInstall + +Setup installation of the main library, public headers and debug symbols. + +
OpmKnown + +Marks as "used" all configuration variables which is used only by some of +the OPM projects, so they don't generate warnings in the rest of them. + +
OpmLibMain + +Driver module for the build process. First reads the list of prerequisites +and options, then set up the compiles and installations. + +
OpmProject + +Set up pkg-config and CMake information files (see [Generated Files] +(#generated-files)) for this project, based on configuration. + +
OpmSatellites + +Build test programs and examples for a library that is bundled in the project. +
+ +### Wrapper Modules + +These modules wrap the CMake `find_library` function and adds the information +retrieved from the imported prerequisite to module-specific variables, so that +these can be added to the build in batch afterwards. + + + + +
File (.cmake)Description
OpmFind + +Wrapper around `find_package`. Searches in various locations relative to this +project as well as in the system directories for a CMake module which can +locate the package. If it is found, adds the project variables (see +[Variables Reference](#variables-reference)) for this project into this one, +for instance include and library directories are added to the compile and link +command-line for this project. + +
OpmPackage + +Typical way of finding an OPM package; searches for the header and library, +and tries to compile a test program. This is the general implementation of +a CMake find module, and is used to locate those of the prerequisites that +fits the pattern. + +
+ +### Configuration Modules + +These are modules for altering the compiler and/or linker option in some way, +or get information from the system. They are not tied to the OPM projects. + + + + + + + + + + + + + + + + + +
File (.cmake)Description
UseCompVer + +Get the version of GCC that is used to compile the project. This is used in +other modules to enable features that are known to exist/work only in certain +versions of GCC. + +
UseDebugSymbols + +Set up the compile to generate debug symbols for the code. This is done also +if a release build was requested, to be able to do post-mortem debugging of +production code. (The debug symbols does not inhibit optimization). + +
UseDuneVer + +Retrieve the version of DUNE which is available. + +
UseDynamicBoost + +Determine if Boost is linked statically or dynamically (shared). This is +necessary to know for the unit tests. + +
UseFastBuilds + +Enable certain techniques which is known to speed up the build itself. + +
UseFortranWrappers + +Provide a macro for declaration of external symbols which is located in +Fortran libraries. It is not necessary to have a Fortran compiler present +to use this macro. + +
UseMultiArch + +Check if the system uses the multi-arch scheme for organizing libraries +(currently derivatives of Debian do this). + +
UseOnlyNeeded + +Only link to libraries which is actually used by the project. Some +platforms provide "under-linked" libraries (they need other libraries +but doesn't state so explicitly, but rather imply that the executable +must link to these itself), and this is also handled. + +
UseOpenMP + +Add OpenMP features to the build. Since OpenMP is activated by pragmas +in the code, compiler options instead of libraries are needed. + +
UseOptimization + +Compile with more extensive optimization that what is the default in +CMake. + +
UsePrecompHeaders + +Set up precompiled headers if the project has a `opm/xxx/opm-xxx-pch.hpp` +header. Due to problems across various compilers, this is currently an +opt-in feature. + +
UseSystemInfo + +Retrieve information about the system the build is performed on. This is +printed in the configuration log and can be helpful to troubleshoot +problems from error reports. + +
UseVCSInfo + +Retrieve information about which Git revision is compiled. This is useful +to figure out which version an error report refers to. + +
UseVersion + +Add version information for this project into the library binary, making +it available for query at runtime. + +
UseWarnings + +Enable a more extensive set of warnings to be reported by the compiler +than what is the default in CMake. + +
+ +## Variables Reference + +The build system will setup variables with names of the pattern `xxx_YYY` +where xxx is the project name (here including the suite; e.g. "opm-core") +and yyy is the suffix in the list below. The project name is used verbatim, +i.e. there is no translation of dashes and case ("opm-core" and not "OPM_CORE"). + + + + + + + + + + + + + + + + + + +
SuffixDescription
_DEFINITIONS + +Macro defines (of the type `-DFOO`) that needs to be added to the compile +of translation units contained in this library. This also includes defines +that must be present to headers which is included by this library. + +
_CONFIG_VARS + +Defines which should be present in `config.h` of the project which +includes this library (client code). Only the names of the variables +are listed here; the actual values must be found by the configuration +script of the client. + +
_CONFIG_IMPL_VARS + +Defines which should be present in `config.h` but is only used by +the internal code of the project itself. Use this list to get +defines without imposing a requirement on client code to also probe +for values. + +
_INCLUDE_DIR + +Directory where the public headers of this project are stored. + +
_INCLUDE_DIRS + +List of include directories that must be on the compiler search +path to compile code which uses this project. In addition to the +headers of this project itself, it also includes the transitive +closure of paths for all prerequisites as well. + +
_LABEL + +Currently for OPM projects, this follows a pattern of `YYYY.MM` +where YYYY is the year of the release and MM is the month. This +gives information to humans about how up to date this instance +of the library is (but doesn't provide a way to check for +compatibility, which is why the VERSION alternatives exist). + +
_LIBRARY + +Name and path of the binary to link with. + +
_LIBRARIES + +Full list of the library of both this project, and all its +prerequisites, that need to be included in the link. I.e. the +client code should only include the transitive list from its +immediate prerequisites and not know about the full dependency +graph. + +
_LIBRARY_DIRS + +Directories that should be added to the linker search path when +including this library. + +
_LINKER_FLAGS + +Flags that must be added to the link when including this library. + +
_SOURCES + +List of source files contained in this project. This enables libraries +to be distributed in source form (e.g. CJSON and TinyXML) and linked +directly into the project. + +
_TARGET + +Name of the library which is generated by this project. CMake and +autotools do not like library names which contains dashes, so they +are stripped out. By using a macro for this we are guaranteed uniform +translation. + +
_VERSION + +Textual concatenation of all components of the version number (see below) +with a dot inbetween. This form of version number can be compared using +CMake VERSION_{LESS|EQUAL|GREATER} operators. + +
_VERSION_MAJOR + +Major version of the library. If the major versions doesn't match, then +compatibility with existing code cannot be reckoned. + +
_VERSION_MINOR + +Minor version of the library. Libraries with newer minor version can +have more features, but should be able to run old code. + +
_VERSION_REVISION + +Micro version of the library. This number is generally incremented +whenever bugfixes or performance improvements are made. +
From e79a65bb810c45d0fd0f2b178aba5a9028c7a99f Mon Sep 17 00:00:00 2001 From: Roland Kaufmann Date: Wed, 15 Jan 2014 09:08:00 +0100 Subject: [PATCH 2/2] Terminology instead of Notation as section header Notation implies that we are going to be given symbols which represents values or such (which arguably could be true since the path is a particular value, and from programming we are used to multi-letter symbols perhaps also with whitespace, but it's a stretch), but Terminology is more accurate, as this section describes fits the definition better: it is a vocabulary of technical terms. Hat tip: @bska --- cmake/OPM-CMake.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cmake/OPM-CMake.md b/cmake/OPM-CMake.md index 06a1183f..18a69e3d 100644 --- a/cmake/OPM-CMake.md +++ b/cmake/OPM-CMake.md @@ -10,7 +10,7 @@ sets up declarative lists of prerequisites and content and rely on convention and pre-made modules to do the build. Its goal is to replace but be compatible with the old autotools-based system. -## Notation +## Terminology In the build system to following abstract locations are referred to: