packSize now uses a reference to map/unordered_map to prevent copying.
In addition we always specify all template parameters for the containers to
allow for sending data if non-default ones are used.
At that stage flow did not set the number of thread in the case where
OMP_NUM_THREADS is not defined. That happens in setupParallelism which
is called when the simulator is run. Hence we have to manually compute
the number of threads similar to there.
When filterConnections_ is called the grid is not load
balanced, yet. Currently that means that grid() will also return the
unbalanced grid and all processes will see the whole global grid.
We will change semantics of the unbalanced grid soon: Only the root
process will see the whole grid and the others will see an empty
partition of it. Hence filtering on this partition will remove all
connections on all wells in the schedule for non-root processes and
produce wrong results.
For non-root process the filtering needs to be done on the load
balanced grid. This is accomplished by this commit.
In the case with active solvent (BlackoilWellModel::has_solvent_ is
true, e.g. case 2D_OILWATER_POLYMER.DATA) and one process have no
wells while another has some this lead to numComponents returning
different values. Later this lead to truncated message in MPI_Allgather.
The underlying problem was the usage of numWells() which returns just
the number of local wells. We resort to using wellsActive which
returns whether there any well active regardless which process knows it.
This removes a deadlock experienced for some models
where we have specified connections to non-active cells.
On non-IO ranks we are using the local grid since in the
future there will be no global grid available. Wells connecting
cells not on these processors are neglected anyway.
Closes#2101
This is needed to support dune-fem where the local mapper might
be
`MultipleCodimMultipleGeomTypeMapper<GridView<GridPart2GridViewTraits<AdaptiveLeafGridPart<CpGrid, (PartitionIteratorType)4,
false> > >, Dune::Impl::MCMGFailLayout>`
as opposed to the global one being
`MultipleCodimMultipleGeomTypeMapper<GridView<DefaultLeafGridViewTraits<CpGrid>>, Dune::Impl::MCMGFailLayout>`.
Closes#2095.
Since the indexMaps do not contain the global element index anymore
(but the global id). The old code did not work anymore.
Unfortunately, we are using CpGrid specific functions (scatterData)
to get the mapping. Therefore this might be broken if other grids are
used.
Previously, it was still assumed that all ranks knew the global grid
and each map on CollectDataToIORank::indexMaps_ was a mapping of
send/receive index to the index of the cell using the mapper of the
corresponding global grid.
With this patch inside of CollectDataToIORank::DistributeIndexMapping
indexMaps is a mapping from send/receive index to global cartesian
index until the destructor is run. Inside of the destructor of the
iorank the remapping to mapped index of the global grid happens and
the ranks array is computed.
This at least slightly improves the old design. In that design the
subclass had no own constructor but inherited the one of the base class.
That base class constructor called certain subclass
functions (createGrids_, filterConnections_, updateOutputDir_, and
finalizeInit_)that would initialize raw pointers of the
subclass. Hence subclasses where not allowed to have non-pod members
and those used later (e.g. deleted in the destructor) had to be
initialized in these functions.
The new (still ugly) design introduces constructors into the
subclasses and skips inheriting constructors. Now one must call a base
class function classImplementationInit which will still call the
functions createGrids_, filterConnections_, updateOutputDir_, and
finalizeInit_, but at least at this point the baseclass is fully
constructed and the subclass is constructed as much as
possible/needed (non-pod types will be initialized now.)
With the change of CPgrid to only holding the grid on one process
it will be an empty grid on all other processes. This has really
strange side effects like Schedule::filterConnections removing all
well perforations on theses processes.
This at least slightly improves the old design. In that design the
subclass had no own constructor but inherited the one of the base class.
That base class constructor called certain subclass
functions (createGrids_, filterConnections_, updateOutputDir_, and
finalizeInit_)that would initialize raw pointers of the
subclass. Hence subclasses where not allowed to have non-pod members
and those used later (e.g. deleted in the destructor) had to be
initialized in these functions.
The new (still ugly) design introduces constructors into the
subclasses and skips inheriting constructors. Now one must call a base
class function classImplementationInit which will still call the
functions createGrids_, filterConnections_, updateOutputDir_, and
finalizeInit_, but at least at this point the baseclass is fully
constructed and the subclass is constructed as much as
possible/needed (non-pod types will be initialized now.)
In the case where two were direct vertical neighbors in the grid
but not in the underlying cartesian grid (e.g. because of MINPV or
pinch outs), we treated them as NNCs and wrote the transmissibilty
to TRANNC.
With this patch we detect this situation (two neighbor cells with identical
i and i and no active cells between them) and do not create an NNC
in the eclipse output files but write the transmissibility to TRANZ.
This change will print the encountered unknown keywords to std::cerr
(albeit in the format ThisKeyword and not --this-keyword) and abort
before running the simulator.
It seems like eclipse ignores NNCs with small transmissibility.
Small means less than 1e-6 for Eclipse (Even if it says that it
is ignoring values below 1e-5 and/or zero values)!.
This commit now implements the same threshold during IO.
Also fixes a bug when applying EDITNNC, it needs to have cell1<=cell2 to work.
Previously the vector of NNCData was passed in as a reference and sorted.
Unfortunately, it needed to be transformed later to meet all prerequisites.
With this commit we do this transformations in sortNncAndApplyEditnnc.
Furthermore EDITNNC data is passed by value as it is not needed
outside and should usually not be to big. It was copied outside anyway!
We first add all NNCs specified in the deck to the ouput
and then determine additional NNCs by iterating over all faces that
connect cells that are not connected in the underlying cartesian grid.
Therefore we need to make sure that we do not output NNCs twice
and for faults that also have a specified NNC we need to substract
the transmissibility specified via NNC.
Unfortunately, we first created NNC with applied EDITNNC and then
still used the original NNC data to set the transmissibility. Thus
we were actually ignoring EDITNNC.
This commit fixes this by using the data structure that has EDITNNC
applied.
There is a problem compiling upstream
Hierarchy::recalculateGalerkin() without MPI. Therefore we
reimplement that method in our AMG version. Fixes will travel
upstream.
Apart from that we made sure that all types have a meaning even
if there is no MPI found and that we at least have redistribute method
to call even if using SequentialInformation. In that case the method
does nothing but prevents compilation errors complaining about missing
parallel functionality.
As the AMG hierarchy is kept and uses the initial OwnerOverlopCommunication object
we need to make sure that this object lives as long as the hierarchy. We assure
this by storing a shared_ptr to it in ISTLSolverEbosCpr and reuse it to construct
the WellModelMatrixAdapter.
Previously the OwnerOverlapCommunication object was owned by the adapter as a
unique_ptr was used. The change to shared_ptr makes it possible to to extract
it and reuse even if OwnerOverlapCommunication is thrown away.
Somehow some standard output when running MILU () is still there and
it really clutters the output and and makes it unusable. This commit
removes it from flow.
This seems to be covered for types and functions by our coding style
with room for interpretation. For variables the coding styles asks for
underlines though, but nevermind.
The former order resulted of first apply NCC to the grid
transmissibilities and then applying EDITNNC resulted in NNCs being
scaled twices. The reason is that applyNNCToGridTrans_ scales the NNC
with EDITNNC. With the patch the order of the function calls is
reversed to prevent double scaling.
This is includes neighboring connection and NNCs due to faults. In both
cases the transmissibilities of specified via NNC are added to the set or
computed ones.
This is the first step for supporting NNC in flow.
Previously, this method did not take parallelism into account but just
checked the local wells for the constraints. Depending on the load balancing
of the wells this sometimes led to different return values of the function on different
processors. As the output is used to limit the time step size, different processors
were sometimes using different time steps in their local computations. This screwed
up convergence checks int the nonlinear operator such that only some processor thought
convergence was already achieved while others wanted to do more iterations.
With this commit the method now returns whether there is any well on any processor
with the constraint being true.
This commit adds an option that allows to enable detecting parallel
logging fallout (option --enable-logging-fallout-warning). It is now
false by default (previous behavior was as if the option was true).
If option is true a warning will be printed for any process with nonzero
rank that does try to log to *.PRT or *.DBG and the logged output will be appended
to these files at the end.
Previously runs of flow started with 'flow base.' Would result in base..DBG
files; note the double dot and the case sensitivity. Now the file will be
BASE.DBG in this case. Note that any extension will be stipped (not only
.DATA as before).
Previously all processes reported
Warning: Fast restart using SAVE is not supported. Standard restart file is written instead.
Now this is done only on the master process where logging is activated.
The problem was only present if UMFPack was not found and therefore
FLOW_SUPPORT_AMG was defined to true. In that case we experienced compile
errors in a source branch that would never be executed. Therefore we remove
the code there and throw an exception.
Previously the regex issued warning for all files containing ".[0-9]*." in the name
in the output directory. That was too general and created more warnings than we wanted.
This fixes fallout from the merge of PR #1512. Since then the
ParallelFileMerger was always given the current directory even if
another output directory was requested. This resulted in
CASENAME.<procid>.<ext> files in that directory never being merged and removed.
Now the ParallelFileMerger gets the output directory path again.
It should be --milu-variant and is it now. Maybe the parameter system
could be a bit smart and detect consecutive upper case letters and treat
them correctly.
Markus Blatt