It seems like the VERSION_GREATER_EQUAL operator for boolean
expressions was introduced after CMake 3.6 and hence the current
check whether to activate CUDA or not is broken in version 3.6 and
below.
This PR fixes this by using VERSION_GREATER.
Closes#2375.
We experienced weired linker errors when using host compiler version for
compilation that were not supported by the nvcc used to compile the
cuda code:
```
[ 15%] Linking CXX executable bin/test_timer
/usr/bin/ld: /home/mblatt/src/dune/opm-2.6/opm-common/opm-seq/lib/libopmcommon.a(Parser.cpp.o): in function `Opm::(anonymous namespace)::file& std::vector<Opm::(anonymous namespace)::file, std::allocator<Opm::(anonymous namespace)::file> >::emplace_back<std::filesystem::__cxx11::path&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&>(std::filesystem::__cxx11::path&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&)':
Parser.cpp:(.text+0x1096): undefined reference to `std::filesystem::__cxx11::path::_M_split_cmpts()'
/usr/bin/ld: Parser.cpp:(.text+0x10ad): undefined reference to `std::filesystem::__cxx11::path::_M_split_cmpts()'
/usr/bin/ld: /home/mblatt/src/dune/opm-2.6/opm-common/opm-seq/lib/libopmcommon.a(Parser.cpp.o): in function `Opm::(anonymous namespace)::ParserState::loadFile(std::filesystem::__cxx11::path const&)':
Parser.cpp:(.text+0x23a1): undefined reference to `std::filesystem::canonical(std::filesystem::__cxx11::path const&)'
/usr/bin/ld: Parser.cpp:(.text+0x24e0): undefined reference to
`std::filesystem::__cxx11::path::_M_split_cmpts()'
```
The reason turned out to be that the library path was build up by
paths of the old (g++-7) compiler used by nvcc and the actual (newer) compiler
g++-8. This completely messed up the linker paths for CMake.
To detect this situation already when running cmake we have resorted
to first setting the CMAKE_CUDA_FLAGS to force cmake to make nvcc use
the host compiler and to activate CUDA (if available) before calling
`find_package(CUDA)`. If the host compiler is not supported CMake will
error out during `enable(CUDA)`
Note that we still use (deprecated) FindCUDA later to determine the
libraries to link to.
The users has the option to either deactivate CUDA by setting
`-DCMAKE_DISABLE_FIND_PACKAGE_CUDA=ON` or to use a compiler supported
by nvcc (setting `-DCMAKE_CXX_COMPILER=compiler`).
Additionally we do not try to activate CUDA the CMake version is <
3.8. Please note that previously CMake would have errored out here
anyway since we used the unsupported `enable_language(CUDA)` even in
this case.
Closes#2363.
This fixes name clashes with DUNE's own factory that is introduced
in DUNE 2.7. Hence it closes issue #2266.
BTW: Dune's factory has more template parameters than ours.
... instead of a reference This is needed as we only want to read the
full deck and construct the EclipseGrid only on the master process
with rank 0. Hence all other ranks will pass a nullptr to this
function. This will be possible with this move from a reference to a
pointer.
packSize now uses a reference to map/unordered_map to prevent copying.
In addition we always specify all template parameters for the containers to
allow for sending data if non-default ones are used.
At that stage flow did not set the number of thread in the case where
OMP_NUM_THREADS is not defined. That happens in setupParallelism which
is called when the simulator is run. Hence we have to manually compute
the number of threads similar to there.
When filterConnections_ is called the grid is not load
balanced, yet. Currently that means that grid() will also return the
unbalanced grid and all processes will see the whole global grid.
We will change semantics of the unbalanced grid soon: Only the root
process will see the whole grid and the others will see an empty
partition of it. Hence filtering on this partition will remove all
connections on all wells in the schedule for non-root processes and
produce wrong results.
For non-root process the filtering needs to be done on the load
balanced grid. This is accomplished by this commit.
In the case with active solvent (BlackoilWellModel::has_solvent_ is
true, e.g. case 2D_OILWATER_POLYMER.DATA) and one process have no
wells while another has some this lead to numComponents returning
different values. Later this lead to truncated message in MPI_Allgather.
The underlying problem was the usage of numWells() which returns just
the number of local wells. We resort to using wellsActive which
returns whether there any well active regardless which process knows it.
This removes a deadlock experienced for some models
where we have specified connections to non-active cells.
On non-IO ranks we are using the local grid since in the
future there will be no global grid available. Wells connecting
cells not on these processors are neglected anyway.
Closes#2101
This is needed to support dune-fem where the local mapper might
be
`MultipleCodimMultipleGeomTypeMapper<GridView<GridPart2GridViewTraits<AdaptiveLeafGridPart<CpGrid, (PartitionIteratorType)4,
false> > >, Dune::Impl::MCMGFailLayout>`
as opposed to the global one being
`MultipleCodimMultipleGeomTypeMapper<GridView<DefaultLeafGridViewTraits<CpGrid>>, Dune::Impl::MCMGFailLayout>`.
Closes#2095.
Since the indexMaps do not contain the global element index anymore
(but the global id). The old code did not work anymore.
Unfortunately, we are using CpGrid specific functions (scatterData)
to get the mapping. Therefore this might be broken if other grids are
used.
Previously, it was still assumed that all ranks knew the global grid
and each map on CollectDataToIORank::indexMaps_ was a mapping of
send/receive index to the index of the cell using the mapper of the
corresponding global grid.
With this patch inside of CollectDataToIORank::DistributeIndexMapping
indexMaps is a mapping from send/receive index to global cartesian
index until the destructor is run. Inside of the destructor of the
iorank the remapping to mapped index of the global grid happens and
the ranks array is computed.
This at least slightly improves the old design. In that design the
subclass had no own constructor but inherited the one of the base class.
That base class constructor called certain subclass
functions (createGrids_, filterConnections_, updateOutputDir_, and
finalizeInit_)that would initialize raw pointers of the
subclass. Hence subclasses where not allowed to have non-pod members
and those used later (e.g. deleted in the destructor) had to be
initialized in these functions.
The new (still ugly) design introduces constructors into the
subclasses and skips inheriting constructors. Now one must call a base
class function classImplementationInit which will still call the
functions createGrids_, filterConnections_, updateOutputDir_, and
finalizeInit_, but at least at this point the baseclass is fully
constructed and the subclass is constructed as much as
possible/needed (non-pod types will be initialized now.)
With the change of CPgrid to only holding the grid on one process
it will be an empty grid on all other processes. This has really
strange side effects like Schedule::filterConnections removing all
well perforations on theses processes.
This at least slightly improves the old design. In that design the
subclass had no own constructor but inherited the one of the base class.
That base class constructor called certain subclass
functions (createGrids_, filterConnections_, updateOutputDir_, and
finalizeInit_)that would initialize raw pointers of the
subclass. Hence subclasses where not allowed to have non-pod members
and those used later (e.g. deleted in the destructor) had to be
initialized in these functions.
The new (still ugly) design introduces constructors into the
subclasses and skips inheriting constructors. Now one must call a base
class function classImplementationInit which will still call the
functions createGrids_, filterConnections_, updateOutputDir_, and
finalizeInit_, but at least at this point the baseclass is fully
constructed and the subclass is constructed as much as
possible/needed (non-pod types will be initialized now.)
In the case where two were direct vertical neighbors in the grid
but not in the underlying cartesian grid (e.g. because of MINPV or
pinch outs), we treated them as NNCs and wrote the transmissibilty
to TRANNC.
With this patch we detect this situation (two neighbor cells with identical
i and i and no active cells between them) and do not create an NNC
in the eclipse output files but write the transmissibility to TRANZ.
This change will print the encountered unknown keywords to std::cerr
(albeit in the format ThisKeyword and not --this-keyword) and abort
before running the simulator.
It seems like eclipse ignores NNCs with small transmissibility.
Small means less than 1e-6 for Eclipse (Even if it says that it
is ignoring values below 1e-5 and/or zero values)!.
This commit now implements the same threshold during IO.
Also fixes a bug when applying EDITNNC, it needs to have cell1<=cell2 to work.
Previously the vector of NNCData was passed in as a reference and sorted.
Unfortunately, it needed to be transformed later to meet all prerequisites.
With this commit we do this transformations in sortNncAndApplyEditnnc.
Furthermore EDITNNC data is passed by value as it is not needed
outside and should usually not be to big. It was copied outside anyway!
We first add all NNCs specified in the deck to the ouput
and then determine additional NNCs by iterating over all faces that
connect cells that are not connected in the underlying cartesian grid.
Therefore we need to make sure that we do not output NNCs twice
and for faults that also have a specified NNC we need to substract
the transmissibility specified via NNC.
Unfortunately, we first created NNC with applied EDITNNC and then
still used the original NNC data to set the transmissibility. Thus
we were actually ignoring EDITNNC.
This commit fixes this by using the data structure that has EDITNNC
applied.
There is a problem compiling upstream
Hierarchy::recalculateGalerkin() without MPI. Therefore we
reimplement that method in our AMG version. Fixes will travel
upstream.
Apart from that we made sure that all types have a meaning even
if there is no MPI found and that we at least have redistribute method
to call even if using SequentialInformation. In that case the method
does nothing but prevents compilation errors complaining about missing
parallel functionality.
As the AMG hierarchy is kept and uses the initial OwnerOverlopCommunication object
we need to make sure that this object lives as long as the hierarchy. We assure
this by storing a shared_ptr to it in ISTLSolverEbosCpr and reuse it to construct
the WellModelMatrixAdapter.
Previously the OwnerOverlapCommunication object was owned by the adapter as a
unique_ptr was used. The change to shared_ptr makes it possible to to extract
it and reuse even if OwnerOverlapCommunication is thrown away.
Somehow some standard output when running MILU () is still there and
it really clutters the output and and makes it unusable. This commit
removes it from flow.
This seems to be covered for types and functions by our coding style
with room for interpretation. For variables the coding styles asks for
underlines though, but nevermind.
The former order resulted of first apply NCC to the grid
transmissibilities and then applying EDITNNC resulted in NNCs being
scaled twices. The reason is that applyNNCToGridTrans_ scales the NNC
with EDITNNC. With the patch the order of the function calls is
reversed to prevent double scaling.
This is includes neighboring connection and NNCs due to faults. In both
cases the transmissibilities of specified via NNC are added to the set or
computed ones.
This is the first step for supporting NNC in flow.
Previously, this method did not take parallelism into account but just
checked the local wells for the constraints. Depending on the load balancing
of the wells this sometimes led to different return values of the function on different
processors. As the output is used to limit the time step size, different processors
were sometimes using different time steps in their local computations. This screwed
up convergence checks int the nonlinear operator such that only some processor thought
convergence was already achieved while others wanted to do more iterations.
With this commit the method now returns whether there is any well on any processor
with the constraint being true.
Markus Blatt