Well rates of distributed wells might be stored on multiple processes
but should be summed only once. Hence only the owner does the
summation with this commit.
This adds an utility that creates a vector of all above values for
the local perforations. For distributed wells this is needed as the
perforation above might live on another processor. We use the parallel
index sets together with the global index of the cells that are
perforated.
The B matrix is basically a component-wise multiplication
with a vector followed by a parallel reduction. We do that
reduction to all ranks computing for the well to save the
broadcast when applying C^T.
BlackoilWellModel now stores an instance of this class for each
well. Inside that class there is a custom communicator that only
contains ranks that will have local cells perforated by the well.
This will be used in the application of the distributed well operator.
This is another small step in the direction of distributed wells,
but it should be safe to merge this (note creation of the custom
communicators is a collective operation in MPI but done only once).
IMHO this might have happened if perf_data_ is empty
or if the last connection is closed. (Discovered while
working on distributed wells but might happen already
before!)
that simplifies the code a bit and will work with
distributed wells. Previously, we assumed that all
non-shut perforations are stored locally. That does
not hold any more.
Only after rank zero does the filtering the schedule the well
definitions in there are guarateed to have no perforations to inactive
cells. Therefore we broadcast the schedule another time to publish
this to all processes.
Previously, we did the filtering locally on these processes bit that
did also remove perforations to cells that are active globally but
not locally. That seems very hard to work with when allowing
distributed wells.
It would remove perforated cells from wells that cross the local
domain's border. That would make it impossible to figure out the
first connection. In addition we would not be able to check that
the connections exist (as rank 0 would have the complete information
-> inconsistency).
In serial we use the first cell of the first well to determine the
pvt region index for a group. Previously, we used the first cell of
the first local well in a parallel run. Unfortunately that may lead
to different pvt region indices being used for the same goup on
different processes.
We fix this by using the same approach in parallel as we already use
in serial. For this we use Well::seqIndex() to determine the needed
ordering.
and use it in the WellInterface instead of creating a vector
with these indices there. The original approach recreates
information in another path of the well and assumes that all
connections are in a process's local partition. That assumption
does not hold any more for distributed wells.
In OPM the matrix graph might be unsymmetric as we do not store
the full sparsity pattern for copy rows but only the diagonal.
Unfortunately, DUNE assumes that matrices from finite elements and
finite volumes have a symmetric sparsity pattern for copy rows to
and uses this assumption to create the graphs for PTScotch/ParMETIS
more easily. But PTScotch/ParMetis assume a symmetric graph.
nvcc exits compilation if the header dune/istl/basearray.hh (form DUNE
2.6) is included as it does not seem to understand the friend declaration
there (friend class for a struct).
```
/usr/include/dune/istl/basearray.hh:101:49: error: ‘typename Dune::Imp::base_array_unmanaged<B, A>::RealIterator’ names ‘template<class B, class A> template<class T> struct Dune::Imp::base_array_unmanaged<B, A>::RealIterator’, which is not a type
friend class RealIterator<const ValueType>;
^
```
Previously, we exported an unordered map containing all names of
wells that are not present in the local part of the grid.
As we envision to have wells that are distributed across multiple
processors, this information does not seem to be enough. We need
to be able to set up communication for each well. To do this we need
to find out who handles perforations of each well.
We now export a full list of well name together with a boolean
indicating whether it perforates local cells (vector of pair of string
and bool).
FiedlPropsManager::keys() list the FieldProperties needed by the
TransCalculator, but these cannot be queried the normal way as this
raises exceptions and results in a deadlock. Hence we use the new
funtionality to get also the unsupported ones, by passing true to
get_double_field_data.
As the ErrorGuard also dumps warnings we now always dump
it (previously only on error) to get these messages in the
console.
If there are error encountered, we log a meaningful error
message (the real cause was missing previously) and do a
graceful exit after MPI_Finalize.
For PINCH(5)==ALL, we take the minimum of MULTZ+ and ignore MULTZ-.
We also prepare for PINCH(5)==TOP taking only the toplevel MULTZ+
value.
For non-vertical directions we use both MULTZ+ and MULTZ-
We used a method isGlobalIdxOnThisRank to determine whether to write
an entry for a summary keyword (like BPR). Unfortunately, this did
exactly what the name suggested, but we actually passed a cartesian
index to it. That meant that a lower cartesian index might have found on
many processes (with different cartesian index and hence resulting in
wrong values), while higher for ones no process would have been found
with it (resulting in writing zeros).
With this commit we store a sorted list of cartesian indices and query
that in the renamed and restructured function isCartesianidxOnThisRank.
Most probably this broke during refactoring.
Closes#2665
This will process the same faces in serial and parallel.
Hence it make the silent assumption that we only process faces
from low cartesian index to high cartesian index hold again.
This should fix PINCH MULTZ ALL in parallel.
These codes are reimplemented in the ebos simulator and should
be reused, instead. This commit factilitates this and starts
reusing the logging setup code in ebos. Hence reduces code duplication.
Markus Blatt