OpenCL header was included and OpenclKernels used unconditionally
in WellContributions.cpp. As this file is compiled if CUDA is found,
too it lead to undefined references for e.g. cl::CommandQueue if
opencl headers were there and compile error if not.
Removed message in amgclbackend was:
warning: catching polymorphic type ‘class
boost::property_tree::json_parser::json_parser_error’ by value
[-Wcatch-value=]
We still request Standard version 1.2 only.
We need to use KernelFunctor instead of make_kernel.
In addition cl::Sources now works on std::string and
does not support std::pair<const char*, in> anymore.
With dune-fem the type of the grid view is
Dune::Fem::AdaptiveLeafGridPart and not the LeafGridView
of the grid. The old approach therefore did not compile
as we passed the wrong view.
This is needed for distributed wells to save most of the code
from checking whether a perforation is in the interior.
We add new methods compressedIndexForInterior that return -1
for non-interior cells and use that for the wells. This restores
the old behaviour before 1cfe3e0aad
We decide whether to write tracer concentrations based on the size
of the tracerConcentrations_ container. Hence we need to make sure
that its size is zero after the concentrations have been written
and now resize the container to reflect that.
Before this change only the container inside tracerConcentrations_
were moved (and hence made empty) and next tiem we passed empty
containers to the output writing functions (which in parallel
triggered an assertion).
As multisegment wells may throw in applyUMFPack this is now needed and
the exception needs to communicated to all processes. We do this in
the linearize method of the well model.
Before this change this is what could happen:
- The process with the exception would have chopped the time step
- The others would have successfully setup the systems and entered the
linear solve
This poduced a deadlock. One processes was waiting in
OPM_END_PARALLEL_TRY during the setup of the shorter time step and in
collective communication during the setup of the linear solver for the
unchopped time step.
This saves some (expensive?) lookups that already have been done
in the well model. We had to make the well_container accessible from
the well model for this.
Using the perforation data will automatically make sure that the
perforations are not shut and reside on this process in a parallel run.
There cannot happen any collective blocking communication within a
parallel try-catch clause if exceptions might be thrown before the
communication. The communication has to either be reached by all
processes or no processes.
Although not declared as such, prepareTimeStep seems to be an internal
function (despite usage in a test) and hence error control can be done
in code calling it.
There was the following problem with the try-catch approach taken:
The calling site `BlackoilWellModel::assemble` looked like this:
```
OPM_BEGIN_PARALLEL_TRY_CATCH();
{
if (iterationIdx == 0) {
calculateExplicitQuantities(local_deferredLogger); // no parallel try-catch
prepareTimeStep(local_deferredLogger); //includes parallel try-catch
}
updateWellControls(local_deferredLogger, /* check group controls */ true);
// Set the well primary variables based on the value of well solutions
initPrimaryVariablesEvaluation();
maybeDoGasLiftOptimize(local_deferredLogger);
assembleWellEq(dt, local_deferredLogger);
}
OPM_END_PARALLEL_TRY_CATCH_LOG(local_deferredLogger, "assemble() failed: ",
terminal_output_);
```
calculateExplicitQuantities had no parallel-try-catch clause inside,
but prepareTimeStep had one.
Unfortunately, calculateExplicitQuantities might throw (on some
processors). In that case non-throwing processors will try to trigger a
collective communication (to check for errors) in
prepareTimeStep. While the one throwing will move to the
OPM_END_PARALLEL_TRY_CATCH_LOG macro at the end and also trigger a different
collective communication. Booom, we have a deadlock.
With this patch there is no (nested parallel)-try-catch clause in the
functions called. (And if an exception is thrown in prepareTimeStep, it
will be logged as being an assemble failure).
The other option would have been to add parallel-try-catch clauses
to all functions called. That would have created a lot more
synchronization points limiting scalability even further.
Not a big fan of Macros but here at least they seem ot be the only
option. The problem is that the catch clauses must all catch the same
exceptions that have a entry in ExceptionType, because they might be
nested. In addition we did not have a catch all clause, which is added
now and is needed in case a called method throws an unexpected exception.
For 10 Million cell problems my compute server (with 128 GB Ram)
starts to swap, when I use debugging tools in parallel runs. I assume
that this might get an issue for others, too.
Now we consistently use unordered_map for the mapping.
Previously, exceptions happening at this stage have deadlocked
flow. E.g. UniformTabulated2DFunction in opm-material throws
a NumericalIssue if the values passed are outside the tabulated
reason. This function is e.g. called in 2-phase CO2-storage cases
during BlackoilModel::initializeWellState
BTW: This is only the first step as it is not very user friendly that
a simulation aborts at this (late) stage.
We got compile errors like:
/home/build/opm-simulators/opm/simulators/linalg/FlexibleSolver1.cpp:24:1: required from here
/usr/include/dune/istl/ilu.hh:140:29: error: 'double' is not a class, struct, or union type
without this patch.
Hence we use the new internal ILU functions if available.
If there are unknown keywords and the parser throws an exception
then we nevertheless broadcasted the eclipseState and schedule.
Unfortunately, these might be null pointers in this case and
the serializer will run into a segmentation fault (e.g. when
serializing the non-existent TableManager)
Broadcasting is now only done if parsing was successful.
Unfortunately, we cannot us the imported targets. They add some compile
parameters using generator expressions based on the CXX_COMPILER_ID.
While we are using the system CXX compiler for most of the stuff, some
cuda code is compiled with nvcc which at least for some versions does
not support -Wno-catch-value (which gets passed as normal compiler
option).
There is no AMGCL_INCLUDE_DIRS when using find_package. We now query
the target amgcl::amgcl for INTERFACE_INCLUDE_DIRS and store the
result in AMGCL_INCLUDE_DIRS.
Note that we cannot link amgcl::amgcl target to libopmsimulators as
this sets the -fopenmp flag for all the source files and makes
compilation with nvcc fail.
Previously, the user had to specify it in the json file read from the
FlexibleSolver or 1 was used. Unfortunately, the index depends on the
model used and it seem rather opaque to a user what that index is.
With this commit we determine the pressure index from the model.
Fixes:
CMake Error at CMakeLists.txt:458 (target_link_libraries):
The keyword signature for target_link_libraries has already been used with
the target "opmsimulators". All uses of target_link_libraries with a
target must be either all-keyword or all-plain.
The uses of the keyword signature are here:
* /var/lib/jenkins/workspace/opm-common-PR-builder/mpi/install/share/opm/cmake/Modules/OpmCompile.cmake:61 (target_link_libraries)
-- Configuring incomplete, errors occurred!
We actually already require at least CMake 2.8.12 due to the embedded
pybind11 (some tests of it are even at 3.0). Anyway as Ubuntu LTS has
3.10.2 I doubt that anything less is tested by us.
If fmtlib is present on the system we used that one
in the normal mode (not header only). Otherwise we
fallback to the embedded one header only.
Searching for the library is done on opm-common.
Executable is named flow_distribute_z and uses the external
loadbalancing information. It can be used to test the distributed
standard wells on SPE9 with 4 or more processes.
You can use EclCpGridVanguard::setExternalLoadBalancer() to
set an external funtion that creates a vector of integers (containing
the partition for each cell) from the grid. If it is set then this
information will be used for loadbalancing, otherwise ZOLTAN.
We introduce a new parameter --enable-distributed-wells=<true|false>
for this. During startup we check that the model either only has
standard wells or that multisegement wells are actively interpreted
as standard wells (by way of passing --enable-multisegment-wells=false
as an option).
This computation is serial and needs a complete representation
of data attached to all preforations (even those stored on
another process). This commit uses the newly created factory to
correctly compute the connection densities for distributed wells.
Some of our computations are heavily serial and need a complete
representation of the data attached to all perforation no matter
whether a perforation lives on the local partition or not. This commit
adds a factory that allows to easily create such a representaion and
helps writing data back to the local representation.
Markus Blatt