[ri-cloud-api]: Add grid data from sumo using api (#14653)

Add grid data from Sumo using ri-cloud-api

Load Eclipse grid data (roff) from Sumo as RimReservoirGridEnsemble, one case per realization, with async fetch of grid properties per timestep.

- New: RiaSumoGrid (transport), RifReaderSumoGridProperty (lazy property reader), RimReservoirGridEnsembleSumo, RimRoffCaseSumo, and a "Create Reservoir Grid Ensemble" command.
- Shared grid: IJK dimensions compared before download; one RigMainGrid is reused across the ensemble when they match, each realization still parses its own ACTNUM before releasing geometry.
- Rename folders: Summary/Sumo -> Summary/Cloud, RimSumoDataSource -> Cloud/ (PDM keywords unchanged, old project files still load).
- UI: cloud data sources also shown in the 3D window's Data Sources dock.
- Fix: realizations no longer lost on project reload; realizations missing data are now reported instead of silently skipped.
This commit is contained in:
Jørgen Herje authored and GitHub committed 2026-09-10 10:33:58 +02:00
1 parent 71c5b5a97d
commit c49f298a7d
70 files changed
+5194 -222

No files matched your search

@@ -1945,7 +1945,15 @@ RiaSumoConnector* RiaApplication::makeSumoConnector()
// The Sumo data is read through the local ri_cloud_api service, so the connector asks that service
// for its address before every request. The authentication parameters still come from the Sumo
// configuration.
auto serverUrlProvider = [this]() { return cloudApiService()->serverUrl(); };
// The address is only usable once the service is actually answering, so make sure it is before
// handing it out. Requests are composed from this, see RiaSumoConnector::server().
auto serverUrlProvider = [this]() -> QString
{
auto* service = cloudApiService();
if ( !service->waitUntilResponding( RiaSumoDefines::serviceReadyTimeoutMillis() ) ) return {};
return service->serverUrl();
};
m_sumoConnector = new RiaSumoConnector( RiuMainWindow::instance(), serverUrlProvider, authority, scopes, clientId, port );
m_sumoConnector->setTokenDataFilePath( RiaSumoDefines::tokenPath() );
@@ -4,6 +4,7 @@ set(SOURCE_GROUP_SOURCE_FILES
${CMAKE_CURRENT_LIST_DIR}/RiaSumoConnector.cpp
${CMAKE_CURRENT_LIST_DIR}/RiaSumoDefines.cpp
${CMAKE_CURRENT_LIST_DIR}/RiaSumoExplore.cpp
${CMAKE_CURRENT_LIST_DIR}/RiaSumoGrid.cpp
${CMAKE_CURRENT_LIST_DIR}/RiaSumoSummary.cpp
${CMAKE_CURRENT_LIST_DIR}/RiaConnectorTools.cpp
${CMAKE_CURRENT_LIST_DIR}/RiaOsduConnector.cpp
@@ -24,6 +24,8 @@
#include <QCoreApplication>
#include <QDir>
#include <QElapsedTimer>
#include <QEventLoop>
#include <QFileInfo>
#include <QHostAddress>
#include <QNetworkAccessManager>
@@ -52,6 +54,10 @@ constexpr int maxConsecutiveFailures = 2;
// Per-request timeout for the health check, kept well below the poll interval.
constexpr int healthCheckTimeoutMs = 5 * 1000;
// Interval between the /alive attempts made while waiting for the service to come up. Much shorter than
// the periodic health check, as a caller is blocked meanwhile.
constexpr int readinessPollIntervalMs = 250;
// Number of ports scanned, starting at the wanted port, when looking for a free port to bind to.
constexpr int portRangeLength = 100;
} // namespace
@@ -229,6 +235,71 @@ void RiaCloudApiService::start()
m_startupTimer.start();
}
//--------------------------------------------------------------------------------------------------
/// The periodic health check first runs a grace period after the launch, which is far too long to make a
/// caller wait for. Poll the service directly instead, so a request can be issued as soon as it answers.
//--------------------------------------------------------------------------------------------------
bool RiaCloudApiService::waitUntilResponding( int timeoutMs )
{
if ( m_isResponding ) return true;
if ( !isRunning() ) start();
QElapsedTimer elapsed;
elapsed.start();
while ( elapsed.elapsed() < timeoutMs )
{
if ( m_port >= 0 )
{
QNetworkRequest request( QUrl( serverUrl() + "/alive" ) );
request.setTransferTimeout( healthCheckTimeoutMs );
QNetworkReply* reply = m_networkAccessManager->get( request );
QEventLoop eventLoop;
QObject::connect( reply, &QNetworkReply::finished, &eventLoop, &QEventLoop::quit );
eventLoop.exec();
const int statusCode = reply->attribute( QNetworkRequest::HttpStatusCodeAttribute ).toInt();
const bool answered = reply->error() == QNetworkReply::NoError && statusCode == 200;
reply->deleteLater();
if ( answered )
{
m_consecutiveFailures = 0;
if ( !m_isResponding )
{
m_isResponding = true;
emit statusChanged();
}
// The service is known to be up, so the grace period before the periodic check has served
// its purpose. Fall back to the ordinary health check from here.
m_startupTimer.stop();
if ( !m_healthTimer.isActive() ) m_healthTimer.start();
return true;
}
}
// The process may still be booting, or may have exited on a missing dependency.
if ( !isRunning() ) break;
QEventLoop delayLoop;
QTimer delayTimer;
delayTimer.setSingleShot( true );
QObject::connect( &delayTimer, &QTimer::timeout, &delayLoop, &QEventLoop::quit );
delayTimer.start( readinessPollIntervalMs );
delayLoop.exec();
}
RiaLogging::error( std::format( "Cloud API service: not responding after {} ms, giving up.", elapsed.elapsed() ) );
return false;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
@@ -59,6 +59,14 @@ public:
// True once the service has answered a health check, i.e. it is actually serving requests.
bool isResponding() const;
// Block until the service answers a health check, starting it when it is not running. Returns
// immediately when it is already responding, and false when it does not answer within the timeout.
//
// Requests must not be issued before this returns true: isRunning() is true as soon as the process is
// launched, while uvicorn may still be booting. Call from the thread owning this object, as the health
// check uses its network access manager.
bool waitUntilResponding( int timeoutMs );
int port() const;
// Base URL of the service, including the port actually in use, e.g. "http://127.0.0.1:8001". Empty
@@ -53,6 +53,7 @@ RiaSumoConnector::RiaSumoConnector( QObject* parent,
: RiaCloudConnector( parent, {}, authority, scopes, clientId, port )
, m_serverUrlProvider( std::move( serverUrlProvider ) )
, m_explore( *this )
, m_grid( *this )
, m_summary( *this )
{
// The transfer thread runs the network requests issued by the blocking wrappers, so the calling thread can
@@ -65,7 +66,11 @@ RiaSumoConnector::RiaSumoConnector( QObject* parent,
QObject::connect( m_transferThread,
&QThread::started,
m_transferContext,
[this]() { m_transferNetworkAccessManager = new QNetworkAccessManager( m_transferContext ); } );
[this]()
{
m_transferNetworkAccessManager = new QNetworkAccessManager( m_transferContext );
m_transferBackgroundNetworkAccessManager = new QNetworkAccessManager( m_transferContext );
} );
// The context object lives on the transfer thread, so let that thread delete it when it stops.
QObject::connect( m_transferThread, &QThread::finished, m_transferContext, &QObject::deleteLater );
@@ -81,6 +86,14 @@ RiaSumoExplore& RiaSumoConnector::explore()
return m_explore;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
RiaSumoGrid& RiaSumoConnector::grid()
{
return m_grid;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
@@ -101,6 +114,22 @@ QString RiaSumoConnector::server() const
return m_serverUrlProvider();
}
//--------------------------------------------------------------------------------------------------
/// The service is started when the access token is granted, and needs a moment before it answers. Ask for
/// the address only once a request is about to be made, so the wait for the service is done here rather
/// than producing a request without a host.
//--------------------------------------------------------------------------------------------------
QString RiaSumoConnector::serviceBaseUrl()
{
const QString baseUrl = server();
if ( baseUrl.isEmpty() )
{
RiaLogging::error( "The local cloud API service is not available, unable to request data from Sumo." );
}
return baseUrl;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
@@ -136,6 +165,20 @@ QNetworkAccessManager* RiaSumoConnector::networkAccessManager()
return m_networkAccessManager;
}
//--------------------------------------------------------------------------------------------------
/// A separate connection pool for background prefetch, see the header comment. Falls back to the ordinary
/// manager off the transfer thread, matching networkAccessManager(), though prefetch always runs there.
//--------------------------------------------------------------------------------------------------
QNetworkAccessManager* RiaSumoConnector::backgroundNetworkAccessManager()
{
if ( m_transferThread && QThread::currentThread() == m_transferThread && m_transferBackgroundNetworkAccessManager )
{
return m_transferBackgroundNetworkAccessManager;
}
return m_networkAccessManager;
}
//--------------------------------------------------------------------------------------------------
/// Run work on the transfer thread and return at once. Nothing is waited for here, so the result has to be
/// delivered by a callback. The token is requested first, while still on the calling thread, because
@@ -198,21 +241,41 @@ void RiaSumoConnector::invokeOnConnectorThread( const std::function<void()>& wor
/// A blob is fetched in two steps, first the pre-signed URI and then the data itself. Both are started here
/// and neither is waited for: each step continues from the reply of the one before.
//--------------------------------------------------------------------------------------------------
void RiaSumoConnector::downloadBlobAsync( const QString& blobId, const std::function<void( const QByteArray& )>& onFinished )
void RiaSumoConnector::downloadBlobAsync( const QString& blobId,
const std::function<void( const QByteArray& )>& onFinished,
int blobTimeoutMillis,
const void* cancelGroup )
{
const QString url = QString( "%1/blobs/%2/sas_token_and_blob_base_uri" ).arg( server() ).arg( blobId );
if ( !onFinished ) return;
// Every exit below reports through onFinished. Returning without calling it would strand whatever is
// waiting for this blob, which can then never be requested again.
const QString baseUrl = serviceBaseUrl();
if ( baseUrl.isEmpty() )
{
RiaLogging::warning( "No Sumo service address available, unable to download blob." );
onFinished( {} );
return;
}
const QString url = QString( "%1/blobs/%2/sas_token_and_blob_base_uri" ).arg( baseUrl ).arg( blobId );
QNetworkRequest networkRequest;
networkRequest.setUrl( QUrl( url ) );
addStandardHeader( networkRequest, transferToken(), RiaCloudDefines::contentTypeJson() );
auto accessInfoReply = networkAccessManager()->get( networkRequest );
abortIfNotFinishedWithin( accessInfoReply, RiaSumoDefines::requestTimeoutMillis() );
auto accessInfoReply = getAndTrackReply( backgroundNetworkAccessManager(), networkRequest, cancelGroup );
// Deliberately not a short deadline. Several of these run at once, the network manager serves only a few
// per host at a time, and the timer runs from when a request is created rather than from when it is
// served, so a short value aborts requests that never got a chance. The blocking path puts no deadline on
// these at all, see downloadBlobs.
abortIfNotFinishedWithin( accessInfoReply, RiaSumoDefines::blobLookupTimeoutMillis() );
QObject::connect( accessInfoReply,
&QNetworkReply::finished,
m_transferContext,
[this, accessInfoReply, blobId, onFinished]()
[this, accessInfoReply, blobId, onFinished, blobTimeoutMillis, cancelGroup]()
{
const QString sasUri = sasUriFromReply( accessInfoReply, blobId );
if ( sasUri.isEmpty() )
@@ -230,8 +293,11 @@ void RiaSumoConnector::downloadBlobAsync( const QString& blobId, const std::func
// added. Do NOT forward the bearer token to the storage host.
blobRequest.setAttribute( QNetworkRequest::RedirectPolicyAttribute, QNetworkRequest::NoLessSafeRedirectPolicy );
auto blobReply = networkAccessManager()->get( blobRequest );
abortIfNotFinishedWithin( blobReply, RiaSumoDefines::requestTimeoutMillis() );
auto blobReply = getAndTrackReply( backgroundNetworkAccessManager(), blobRequest, cancelGroup );
// A deadline on the transfer itself, so it is only applied when the caller asked
// for one. A large blob on a slow link is not a failure.
if ( blobTimeoutMillis != noTimeout() ) abortIfNotFinishedWithin( blobReply, blobTimeoutMillis );
QObject::connect( blobReply,
&QNetworkReply::finished,
@@ -240,6 +306,53 @@ void RiaSumoConnector::downloadBlobAsync( const QString& blobId, const std::func
} );
}
//--------------------------------------------------------------------------------------------------
/// Create a reply and register it for cancelGroup as one atomic step. Held under the same mutex cancelGroup
/// locks, so a cancelGroup call from another thread either completes fully before this reply exists (in
/// which case it correctly leaves this new, later request alone) or fully after it is tracked (in which case
/// it finds and aborts it as usual): there is no gap in between where the reply exists but is not yet
/// visible to cancelGroup.
//--------------------------------------------------------------------------------------------------
QNetworkReply*
RiaSumoConnector::getAndTrackReply( QNetworkAccessManager* networkManager, const QNetworkRequest& networkRequest, const void* groupId )
{
QMutexLocker locker( &m_activeRepliesMutex );
QNetworkReply* reply = networkManager->get( networkRequest );
if ( groupId && reply ) m_activeReplies.emplace( groupId, reply );
return reply;
}
//--------------------------------------------------------------------------------------------------
/// Abort every reply still tracked under groupId. Replies already finished (and thus removed from the
/// registry by their QPointer clearing) are not touched, only the ones still in flight.
//--------------------------------------------------------------------------------------------------
void RiaSumoConnector::cancelGroup( const void* groupId )
{
if ( !groupId ) return;
std::vector<QPointer<QNetworkReply>> repliesToAbort;
{
QMutexLocker locker( &m_activeRepliesMutex );
auto range = m_activeReplies.equal_range( groupId );
for ( auto it = range.first; it != range.second; ++it )
{
if ( it->second ) repliesToAbort.push_back( it->second );
}
m_activeReplies.erase( range.first, range.second );
}
for ( auto& reply : repliesToAbort )
{
if ( !reply ) continue;
// abort() must run on the thread the reply lives on. QMetaObject::invokeMethod with the default auto
// connection marshals the call there when this is a different thread, and calls it directly otherwise.
QMetaObject::invokeMethod( reply, &QNetworkReply::abort );
}
}
//--------------------------------------------------------------------------------------------------
/// Nothing waits on an async reply, so a request that never answers would otherwise keep its data pending
/// for good. Aborting makes the reply finish with an error, which the chain reports as a failed transfer.
@@ -316,7 +429,7 @@ void RiaSumoConnector::runOnTransferThreadBlocking( const std::function<void()>&
//--------------------------------------------------------------------------------------------------
/// Wait until every reply has finished, or the timeout expires.
//--------------------------------------------------------------------------------------------------
void RiaSumoConnector::waitForRepliesToFinish( const std::vector<QNetworkReply*>& replies )
void RiaSumoConnector::waitForRepliesToFinish( const std::vector<QNetworkReply*>& replies, int perReplyTimeoutMillis )
{
if ( replies.empty() ) return;
@@ -348,7 +461,7 @@ void RiaSumoConnector::waitForRepliesToFinish( const std::vector<QNetworkReply*>
// given on its own. A batch must not be more likely to time out than the same requests made one
// by one, and the server can take a while to answer: a summary vector that has not been
// aggregated yet is produced on demand by the first request that asks for it.
timer.start( static_cast<int>( replies.size() ) * RiaSumoDefines::requestTimeoutMillis() );
timer.start( static_cast<int>( replies.size() ) * perReplyTimeoutMillis );
eventLoop.exec();
}
@@ -361,9 +474,9 @@ void RiaSumoConnector::waitForRepliesToFinish( const std::vector<QNetworkReply*>
//--------------------------------------------------------------------------------------------------
/// Download one blob and return its contents. The download and the wait for it run on the transfer thread.
//--------------------------------------------------------------------------------------------------
QByteArray RiaSumoConnector::downloadBlobBlocking( const QString& blobId )
QByteArray RiaSumoConnector::downloadBlobBlocking( const QString& blobId, const QString& description )
{
const auto contentsByBlobId = downloadBlobsBlocking( { blobId } );
const auto contentsByBlobId = downloadBlobsBlocking( { blobId }, description );
if ( auto it = contentsByBlobId.find( blobId ); it != contentsByBlobId.end() ) return it->second;
@@ -374,10 +487,18 @@ QByteArray RiaSumoConnector::downloadBlobBlocking( const QString& blobId )
/// Issue a GET and return the response body, waiting on the transfer thread. Returns an empty array when
/// the request fails. This is the primitive the data specific requests are built from.
//--------------------------------------------------------------------------------------------------
QByteArray RiaSumoConnector::getBlocking( const QString& url, const QString& progressText )
QByteArray RiaSumoConnector::getBlocking( const QString& path, const QString& progressText )
{
requestTokenBlocking();
// Resolved here and not at the call site: requesting the token is what starts the service, so the
// address only exists from this point on. Done before the work is handed to the transfer thread, as
// waiting for the service uses the network access manager of the calling thread.
const QString baseUrl = serviceBaseUrl();
if ( baseUrl.isEmpty() ) return {};
const QString url = baseUrl + path;
QByteArray body;
runOnTransferThreadBlocking(
@@ -447,16 +568,21 @@ QByteArray RiaSumoConnector::replyBody( QNetworkReply* reply, const QString& url
/// the calling thread dispatches no events while the transfers are in flight. See downloadBlobs for what
/// the transfers actually are.
//--------------------------------------------------------------------------------------------------
std::map<QString, QByteArray> RiaSumoConnector::downloadBlobsBlocking( const std::vector<QString>& blobIds )
std::map<QString, QByteArray> RiaSumoConnector::downloadBlobsBlocking( const std::vector<QString>& blobIds, const QString& description )
{
if ( blobIds.empty() ) return {};
requestTokenBlocking();
const QString baseUrl = serviceBaseUrl();
if ( baseUrl.isEmpty() ) return {};
std::map<QString, QByteArray> contentsByBlobId;
runOnTransferThreadBlocking( [&]() { contentsByBlobId = downloadBlobs( blobIds ); },
QString( "Downloading %1 file(s) from Sumo" ).arg( blobIds.size() ) );
const QString progressText = description.isEmpty() ? QString( "Downloading %1 file(s) from Sumo" ).arg( blobIds.size() )
: QString( "Downloading %1 from Sumo" ).arg( description );
runOnTransferThreadBlocking( [&]() { contentsByBlobId = downloadBlobs( baseUrl, blobIds ); }, progressText );
return contentsByBlobId;
}
@@ -469,7 +595,7 @@ std::map<QString, QByteArray> RiaSumoConnector::downloadBlobsBlocking( const std
///
/// Always called on the transfer thread, where the event loops it waits on dispatch no GUI events.
//--------------------------------------------------------------------------------------------------
std::map<QString, QByteArray> RiaSumoConnector::downloadBlobs( const std::vector<QString>& blobIds )
std::map<QString, QByteArray> RiaSumoConnector::downloadBlobs( const QString& baseUrl, const std::vector<QString>& blobIds )
{
std::map<QString, QByteArray> contentsByBlobId;
if ( blobIds.empty() ) return contentsByBlobId;
@@ -478,7 +604,7 @@ std::map<QString, QByteArray> RiaSumoConnector::downloadBlobs( const std::vector
std::vector<QNetworkReply*> accessInfoReplies;
for ( const auto& blobId : blobIds )
{
QString url = QString( "%1/blobs/%2/sas_token_and_blob_base_uri" ).arg( server() ).arg( blobId );
QString url = QString( "%1/blobs/%2/sas_token_and_blob_base_uri" ).arg( baseUrl ).arg( blobId );
QNetworkRequest networkRequest;
networkRequest.setUrl( QUrl( url ) );
@@ -516,7 +642,11 @@ std::map<QString, QByteArray> RiaSumoConnector::downloadBlobs( const std::vector
blobIndices.push_back( i );
}
waitForRepliesToFinish( blobReplies );
// Phase 2 moves the blob data itself, potentially a large grid or property array, and must not share the
// short deadline sized for the metadata lookup above: under load from other concurrent transfers (e.g.
// another realization's property prefetch), a real transfer can easily take longer than that. Give it the
// same generous deadline the async grid property path uses for its own blob transfer.
waitForRepliesToFinish( blobReplies, RiaSumoDefines::gridPropertyTransferTimeoutMillis() );
for ( size_t i = 0; i < blobReplies.size(); i++ )
{
@@ -21,11 +21,13 @@
#include "RiaCloudConnector.h"
#include "RiaSumoDefines.h"
#include "RiaSumoExplore.h"
#include "RiaSumoGrid.h"
#include "RiaSumoSummary.h"
#include <QByteArray>
#include <QMutex>
#include <QNetworkAccessManager>
#include <QPointer>
#include <QtNetworkAuth/QOAuth2AuthorizationCodeFlow>
#include <functional>
@@ -59,18 +61,30 @@ public:
// Download blobs by id and return their contents. Getting a blob takes two round trips, one for the
// pre-signed URI and one for the data, and a batch does each of those as one concurrent group.
QByteArray downloadBlobBlocking( const QString& blobId );
std::map<QString, QByteArray> downloadBlobsBlocking( const std::vector<QString>& blobIds );
// The description names what is being fetched in the progress dialog, e.g. "grid DROGON realization 1".
// Falls back to a plain file count when it is empty.
QByteArray downloadBlobBlocking( const QString& blobId, const QString& description = {} );
std::map<QString, QByteArray> downloadBlobsBlocking( const std::vector<QString>& blobIds, const QString& description = {} );
// What Sumo holds: assets, cases, ensembles and realizations.
RiaSumoExplore& explore();
// The grid data of a case. Owned here so its blob cache lives as long as the connection.
RiaSumoGrid& grid();
// The summary data of a case.
RiaSumoSummary& summary();
// Transport used by the data specific delegates. Every request goes through the transfer thread, so
// the calling thread waits without dispatching events.
QByteArray getBlocking( const QString& url, const QString& progressText = {} );
// Issue a GET against the local cloud API service and return the response body. Takes the path only,
// e.g. "/cases/{id}/ensembles": the base URL is prepended here, once the service is known to answer.
// Composing the full URL at the call site would capture the address before the service has one.
QByteArray getBlocking( const QString& path, const QString& progressText = {} );
// The base URL of the local cloud API service, empty when it could not be made ready. May block while
// the service starts, so call it from the calling thread and not from the transfer thread.
QString serviceBaseUrl();
// The REST API returns a blob id as a plain string, quoted by FastAPI.
static QString blobIdFromBody( const QByteArray& body );
@@ -90,25 +104,64 @@ public:
// async work are delivered through this, so a caller never has its data handed to it on another thread.
void invokeOnConnectorThread( const std::function<void()>& work );
// Download one blob, calling onFinished with its contents. Call on the transfer thread, where onFinished
// is called as well. Empty contents mean the transfer failed.
void downloadBlobAsync( const QString& blobId, const std::function<void( const QByteArray& )>& onFinished );
// Download one blob. Call on the transfer thread, where onFinished is called as well, exactly once on
// every path. Empty contents mean the transfer failed.
//
// blobTimeoutMillis is a deadline on the transfer itself, not an inactivity timeout: pass noTimeout() for
// a blob large enough that a slow link is not a failure. The blob id request keeps its own deadline.
//
// cancelGroup lets the owner of the async request abort it early, see getAndTrackReply/cancelGroup: pass
// the requesting object's lifetime token so its transfers are aborted, not just ignored, once it is gone.
void downloadBlobAsync( const QString& blobId,
const std::function<void( const QByteArray& )>& onFinished,
int blobTimeoutMillis = RiaSumoDefines::requestTimeoutMillis(),
const void* cancelGroup = nullptr );
// Issue a GET request and register the reply for cancelGroup in one atomic step, guarded by the same
// mutex cancelGroup locks. Creating the reply and tracking it as two separate calls left a gap in which
// a cancelGroup call landing on another thread in between would not find the reply yet, and miss it.
// groupId identifies the owner requesting the transfer, typically its m_lifetimeToken.get(); a null
// groupId tracks nothing. The reply is dropped from the registry automatically once it is deleted.
QNetworkReply* getAndTrackReply( QNetworkAccessManager* networkManager, const QNetworkRequest& networkRequest, const void* groupId );
// Abort every reply still tracked under groupId. Called when the owner of an async request (a reader or
// an ensemble) is destroyed, so its in-flight transfers stop consuming connections and logging results
// for an object that is no longer there, instead of running until their own timeout.
void cancelGroup( const void* groupId );
// Timeout value meaning "never abort on a deadline".
static constexpr int noTimeout() { return -1; }
// Abort a reply that has not finished in time, so an async chain reports a failure instead of hanging and
// leaving whoever waits for the data waiting forever.
static void abortIfNotFinishedWithin( QNetworkReply* reply, int timeoutMillis );
// The network manager belonging to the calling thread: the transfer thread manager when called from
// there, otherwise the one owned by RiaCloudConnector on the GUI thread.
// there, otherwise the one owned by RiaCloudConnector on the GUI thread. Used for interactive requests:
// opening a case, the grid geometry and metadata, and any request the user is waiting on.
QNetworkAccessManager* networkAccessManager();
// A second network manager on the transfer thread, used only by the background prefetch paths
// (downloadBlobAsync and the blob id requests feeding it). Qt limits concurrent connections per host
// separately for each QNetworkAccessManager, so giving prefetch its own manager keeps a burst of
// look-ahead property transfers from starving an interactive request for a different realization or
// case, which would otherwise queue behind them and fail on its own short deadline.
QNetworkAccessManager* backgroundNetworkAccessManager();
// The token for a request issued from the transfer thread. token() reads objects owned by another thread.
QString transferToken() const;
static void waitForRepliesToFinish( const std::vector<QNetworkReply*>& replies );
// perReplyTimeoutMillis is the deadline given to each individual reply were it waited on alone; the group
// as a whole is allowed replies.size() times that, see the implementation. Defaults to requestTimeoutMillis,
// right for the small metadata requests (e.g. the SAS token lookup); pass a longer value for a phase that
// moves real data, such as the blob transfer itself, so it is not cut off by a deadline sized for a lookup.
static void waitForRepliesToFinish( const std::vector<QNetworkReply*>& replies,
int perReplyTimeoutMillis = RiaSumoDefines::requestTimeoutMillis() );
// Issue and collect the two round trips a blob transfer needs. Called on the transfer thread.
std::map<QString, QByteArray> downloadBlobs( const std::vector<QString>& blobIds );
// Runs on the transfer thread, so the base URL is resolved by the caller and passed in: waiting for the
// service must not happen here.
std::map<QString, QByteArray> downloadBlobs( const QString& baseUrl, const std::vector<QString>& blobIds );
public slots:
void requestFailed( const QAbstractOAuth::Error error );
@@ -127,6 +180,7 @@ private:
std::function<QString()> m_serverUrlProvider;
RiaSumoExplore m_explore;
RiaSumoGrid m_grid;
RiaSumoSummary m_summary;
// Transfers run on their own thread so the calling thread can wait without dispatching events. Waiting on
@@ -137,7 +191,16 @@ private:
QObject* m_transferContext = nullptr; // lives on the transfer thread
QNetworkAccessManager* m_transferNetworkAccessManager = nullptr; // created on the transfer thread
// A second manager on the transfer thread, dedicated to background prefetch transfers, see
// backgroundNetworkAccessManager().
QNetworkAccessManager* m_transferBackgroundNetworkAccessManager = nullptr; // created on the transfer thread
// Written on the thread handing work over, read on the transfer thread running it.
mutable QMutex m_transferTokenMutex;
QString m_transferToken;
// Replies tracked so cancelGroup can abort them, keyed by the owner's group id. QPointer clears itself
// when a reply is deleted, so a finished transfer needs no explicit removal beyond that.
QMutex m_activeRepliesMutex;
std::multimap<const void*, QPointer<QNetworkReply>> m_activeReplies;
};
@@ -44,3 +44,27 @@ int RiaSumoDefines::asyncRequestTimeoutMillis()
{
return 5 * 60 * 1000;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
int RiaSumoDefines::blobLookupTimeoutMillis()
{
return 2 * 60 * 1000;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
int RiaSumoDefines::gridPropertyTransferTimeoutMillis()
{
return 5 * 60 * 1000;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
int RiaSumoDefines::serviceReadyTimeoutMillis()
{
return 20 * 1000;
}
@@ -36,4 +36,17 @@ int requestTimeoutMillis();
// given up on, and generous because a summary vector that has not been aggregated yet is produced on demand
// by the request asking for it. Nothing is blocked while it runs, so waiting longer costs nothing.
int asyncRequestTimeoutMillis();
// Deadline for the small requests that resolve which blob to fetch and where it lives. Shorter than the
// transfer that follows: they move almost no data, so taking minutes means the answer is not coming.
int blobLookupTimeoutMillis();
// Deadline for one grid property transfer. Long enough that a large blob on a slow link is not cut off,
// and short enough that a stalled transfer fails instead of leaving the time step blank for the rest of
// the session. A failed step is not retried, so erring on the generous side is the cheaper mistake.
int gridPropertyTransferTimeoutMillis();
// How long to wait for the local cloud API service to answer before giving up on a request. Long enough
// to cover a cold uvicorn boot, short enough that a service that will never come up does not hang the UI.
int serviceReadyTimeoutMillis();
}; // namespace RiaSumoDefines
@@ -40,9 +40,7 @@ RiaSumoExplore::RiaSumoExplore( RiaSumoConnector& connector )
//--------------------------------------------------------------------------------------------------
std::vector<SumoAsset> RiaSumoExplore::assets()
{
const QString url = QString( "%1/assets" ).arg( m_connector.server() );
return parseAssets( m_connector.getBlocking( url, "Loading assets from Sumo" ) );
return parseAssets( m_connector.getBlocking( "/assets", "Loading assets from Sumo" ) );
}
//--------------------------------------------------------------------------------------------------
@@ -50,9 +48,9 @@ std::vector<SumoAsset> RiaSumoExplore::assets()
//--------------------------------------------------------------------------------------------------
std::vector<SumoCase> RiaSumoExplore::cases( const QString& assetName )
{
const QString url = QString( "%1/cases?asset_name=%2" ).arg( m_connector.server() ).arg( QString( QUrl::toPercentEncoding( assetName ) ) );
const QString path = QString( "/cases?asset_name=%1" ).arg( QString( QUrl::toPercentEncoding( assetName ) ) );
return parseCases( m_connector.getBlocking( url, QString( "Loading the cases of %1 from Sumo" ).arg( assetName ) ) );
return parseCases( m_connector.getBlocking( path, QString( "Loading the cases of %1 from Sumo" ).arg( assetName ) ) );
}
//--------------------------------------------------------------------------------------------------
@@ -60,9 +58,9 @@ std::vector<SumoCase> RiaSumoExplore::cases( const QString& assetName )
//--------------------------------------------------------------------------------------------------
std::vector<QString> RiaSumoExplore::ensembleNames( const SumoCaseId& caseId )
{
const QString url = QString( "%1/cases/%2/ensembles" ).arg( m_connector.server() ).arg( caseId.get() );
const QString path = QString( "/cases/%1/ensembles" ).arg( caseId.get() );
return parseEnsembleNames( m_connector.getBlocking( url, "Loading ensembles from Sumo" ) );
return parseEnsembleNames( m_connector.getBlocking( path, "Loading ensembles from Sumo" ) );
}
//--------------------------------------------------------------------------------------------------
@@ -72,10 +70,9 @@ std::vector<QString> RiaSumoExplore::realizationIds( const SumoCaseId& caseId, c
{
const QString encodedEnsembleName = QUrl::toPercentEncoding( ensembleName );
const QString url =
QString( "%1/cases/%2/ensembles/%3/realizations" ).arg( m_connector.server() ).arg( caseId.get() ).arg( encodedEnsembleName );
const QString path = QString( "/cases/%1/ensembles/%2/realizations" ).arg( caseId.get() ).arg( encodedEnsembleName );
return parseRealizationIds( m_connector.getBlocking( url, "Loading realizations from Sumo" ) );
return parseRealizationIds( m_connector.getBlocking( path, "Loading realizations from Sumo" ) );
}
//--------------------------------------------------------------------------------------------------
@@ -0,0 +1,558 @@
/////////////////////////////////////////////////////////////////////////////////
//
// Copyright (C) 2026- Equinor ASA
//
// ResInsight is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// ResInsight is distributed in the hope that it will be useful, but WITHOUT ANY
// WARRANTY; without even the implied warranty of MERCHANTABILITY or
// FITNESS FOR A PARTICULAR PURPOSE.
//
// See the GNU General Public License at <http://www.gnu.org/licenses/gpl.html>
// for more details.
//
/////////////////////////////////////////////////////////////////////////////////
#include "RiaSumoGrid.h"
#include "RiaCloudDefines.h"
#include "RiaLogging.h"
#include "RiaSumoConnector.h"
#include "RiaSumoDefines.h"
#include <QEventLoop>
#include <QJsonArray>
#include <QJsonDocument>
#include <QJsonObject>
#include <QNetworkAccessManager>
#include <QNetworkReply>
#include <QTimer>
#include <QUrl>
#include <algorithm>
#include <optional>
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
std::vector<int> SumoGridInfo::realizationIds() const
{
std::vector<int> ids;
ids.reserve( realizationInfos.size() );
for ( const auto& realizationInfo : realizationInfos )
{
ids.push_back( realizationInfo.realization );
}
return ids;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
bool SumoGridInfo::hasIdenticalDimensions( const std::vector<int>& realizations ) const
{
if ( realizations.empty() ) return false;
std::optional<SumoGridDimensions> referenceDimensions;
for ( int realization : realizations )
{
auto it = std::ranges::find( realizationInfos, realization, &SumoGridRealizationInfo::realization );
// A realization this grid does not have, or one the server reported no dimensions for, makes the
// comparison inconclusive. Report not identical, so a grid is loaded per realization.
if ( it == realizationInfos.end() || !it->dimensions.isValid() ) return false;
if ( !referenceDimensions )
{
referenceDimensions = it->dimensions;
}
else if ( it->dimensions != *referenceDimensions )
{
return false;
}
}
return true;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
RiaSumoGrid::RiaSumoGrid( RiaSumoConnector& connector )
: m_connector( connector )
{
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
std::vector<QString> RiaSumoGrid::gridNames( const SumoCaseId& caseId, const QString& ensembleName )
{
const QString encodedEnsembleName = QUrl::toPercentEncoding( ensembleName );
const QString path = QString( "/cases/%1/ensembles/%2/grid_names" ).arg( caseId.get() ).arg( encodedEnsembleName );
return parseGridNames( m_connector.getBlocking( path ) );
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
SumoGridInfo RiaSumoGrid::gridInfo( const SumoCaseId& caseId, const QString& ensembleName, const QString& gridName )
{
const QString encodedEnsembleName = QUrl::toPercentEncoding( ensembleName );
const QString encodedGridName = QUrl::toPercentEncoding( gridName );
const QString path = QString( "/cases/%1/ensembles/%2/grid_info/%3" ).arg( caseId.get() ).arg( encodedEnsembleName ).arg( encodedGridName );
auto gridInfo = parseGridInfo( m_connector.getBlocking( path ) );
// An empty result also means the request failed: the response body is discarded on any HTTP error.
RiaLogging::debug( std::format( "Grid '{}' realization count : {}", gridName.toStdString(), gridInfo.realizationInfos.size() ) );
return gridInfo;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
QByteArray RiaSumoGrid::gridData( const SumoCaseId& caseId, const QString& ensembleName, const QString& gridName, int realization )
{
const QString blobId = gridBlobId( caseId, ensembleName, gridName, realization );
if ( blobId.isEmpty() ) return {};
QByteArray contents = m_connector.downloadBlobBlocking( blobId, QString( "grid %1 realization %2" ).arg( gridName ).arg( realization ) );
// Every realization downloads its own grid, also in shared grid mode, as the roff blob is the only
// source of the active cells of a realization. Sharing a grid saves memory, not transfers.
RiaLogging::debug( std::format( "Downloaded grid '{}' realization {}: {} bytes", gridName.toStdString(), realization, contents.size() ) );
return contents;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
QString RiaSumoGrid::gridBlobId( const SumoCaseId& caseId, const QString& ensembleName, const QString& gridName, int realization )
{
const QString encodedEnsembleName = QUrl::toPercentEncoding( ensembleName );
const QString encodedGridName = QUrl::toPercentEncoding( gridName );
const QString path = QString( "/cases/%1/ensembles/%2/grids/%3/realizations/%4/blob_id" )
.arg( caseId.get() )
.arg( encodedEnsembleName )
.arg( encodedGridName )
.arg( realization );
return blobIdFromBody( m_connector.getBlocking( path ), "grid", gridName );
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
std::vector<SumoGridPropertyInfo>
RiaSumoGrid::propertyInfo( const SumoCaseId& caseId, const QString& ensembleName, const QString& gridName, int realization )
{
const QString encodedEnsembleName = QUrl::toPercentEncoding( ensembleName );
const QString encodedGridName = QUrl::toPercentEncoding( gridName );
const QString path = QString( "/cases/%1/ensembles/%2/grids/%3/realizations/%4/property_info_list" )
.arg( caseId.get() )
.arg( encodedEnsembleName )
.arg( encodedGridName )
.arg( realization );
return parsePropertyInfo( m_connector.getBlocking( path ) );
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
QByteArray RiaSumoGrid::propertyData( const SumoCaseId& caseId,
const QString& ensembleName,
const QString& gridName,
int realization,
const QString& propertyName,
const QString& isoDateOrInterval )
{
const QString blobId = propertyBlobId( caseId, ensembleName, gridName, realization, propertyName, isoDateOrInterval );
if ( blobId.isEmpty() ) return {};
const QString description = isoDateOrInterval.isEmpty()
? QString( "%1 realization %2" ).arg( propertyName ).arg( realization )
: QString( "%1 %2 realization %3" ).arg( propertyName, isoDateOrInterval ).arg( realization );
return m_connector.downloadBlobBlocking( blobId, description );
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
std::map<QString, QByteArray> RiaSumoGrid::propertyDataBatch( const SumoCaseId& caseId,
const QString& ensembleName,
const QString& gridName,
int realization,
const QString& propertyName,
const std::vector<QString>& isoDatesOrIntervals )
{
// Drop duplicates so a time step is never requested twice in the same batch.
std::vector<QString> timestampsToFetch;
for ( const auto& isoDateOrInterval : isoDatesOrIntervals )
{
if ( std::ranges::find( timestampsToFetch, isoDateOrInterval ) != timestampsToFetch.end() ) continue;
timestampsToFetch.push_back( isoDateOrInterval );
}
if ( timestampsToFetch.empty() ) return {};
// The requests below are issued directly on the transfer thread, so the token and the address of the
// service have to be in place before the work is handed over.
m_connector.requestTokenBlocking();
const QString baseUrl = m_connector.serviceBaseUrl();
if ( baseUrl.isEmpty() ) return {};
std::map<QString, QByteArray> contentsByTimestamp;
m_connector.runOnTransferThreadBlocking(
[&]()
{
contentsByTimestamp = fetchPropertyBatch( baseUrl, caseId, ensembleName, gridName, realization, propertyName, timestampsToFetch );
},
QString( "Loading %1 time step(s) of %2 from Sumo" ).arg( timestampsToFetch.size() ).arg( propertyName ) );
return contentsByTimestamp;
}
//--------------------------------------------------------------------------------------------------
/// The async twin of propertyDataBatch, built on the same blob id request. Every time step is requested at
/// once and delivered on the connector thread as it arrives, so nothing waits for the slowest one.
//--------------------------------------------------------------------------------------------------
void RiaSumoGrid::propertyDataBatchAsync( const SumoCaseId& caseId,
const QString& ensembleName,
const QString& gridName,
int realization,
const QString& propertyName,
const std::vector<QString>& isoDatesOrIntervals,
const std::function<void( const QString&, const QByteArray& )>& onTimeStepReady,
const void* cancelGroup )
{
if ( isoDatesOrIntervals.empty() || !onTimeStepReady ) return;
// Resolved here and not on the transfer thread: requesting the token is what starts the service, and
// waiting for it uses the network access manager of the calling thread.
const QString baseUrl = m_connector.serviceBaseUrl();
if ( baseUrl.isEmpty() )
{
// Report every time step as failed rather than returning silently, so the caller is not left waiting
// for replies that will never come.
for ( const auto& isoDateOrInterval : isoDatesOrIntervals )
{
m_connector.invokeOnConnectorThread( [onTimeStepReady, isoDateOrInterval]() { onTimeStepReady( isoDateOrInterval, {} ); } );
}
return;
}
m_connector.runOnTransferThread(
[this, baseUrl, caseId, ensembleName, gridName, realization, propertyName, isoDatesOrIntervals, onTimeStepReady, cancelGroup]()
{
for ( const auto& isoDateOrInterval : isoDatesOrIntervals )
{
auto deliver = [this, onTimeStepReady, isoDateOrInterval]( const QByteArray& contents )
{
m_connector.invokeOnConnectorThread( [onTimeStepReady, isoDateOrInterval, contents]()
{ onTimeStepReady( isoDateOrInterval, contents ); } );
};
auto blobIdReply = makePropertyBlobIdRequest( baseUrl,
caseId,
ensembleName,
gridName,
realization,
propertyName,
isoDateOrInterval,
m_connector.backgroundNetworkAccessManager(),
cancelGroup );
if ( !blobIdReply )
{
deliver( {} );
continue;
}
// Shorter than the transfer it precedes: this one moves almost no data, so minutes really
// does mean broken.
RiaSumoConnector::abortIfNotFinishedWithin( blobIdReply, RiaSumoDefines::blobLookupTimeoutMillis() );
QObject::connect( blobIdReply,
&QNetworkReply::finished,
blobIdReply,
[this, blobIdReply, propertyName, deliver, cancelGroup]()
{
const QString blobId = blobIdFromReply( blobIdReply, propertyName );
if ( blobId.isEmpty() )
{
deliver( {} );
return;
}
m_connector.downloadBlobAsync( blobId,
deliver,
RiaSumoDefines::gridPropertyTransferTimeoutMillis(),
cancelGroup );
} );
}
} );
}
//--------------------------------------------------------------------------------------------------
/// Resolve the blob ids of a batch of time steps and download the blobs. Always called on the transfer
/// thread, where the event loops it waits on dispatch no GUI events. The contents are returned rather
/// than retained, so the caller decides what to keep.
//--------------------------------------------------------------------------------------------------
std::map<QString, QByteArray> RiaSumoGrid::fetchPropertyBatch( const QString& baseUrl,
const SumoCaseId& caseId,
const QString& ensembleName,
const QString& gridName,
int realization,
const QString& propertyName,
const std::vector<QString>& timestampsToFetch )
{
// Phase 1: resolve all blob ids concurrently.
std::vector<QNetworkReply*> blobIdReplies;
for ( const auto& isoDateOrInterval : timestampsToFetch )
{
blobIdReplies.push_back( makePropertyBlobIdRequest( baseUrl,
caseId,
ensembleName,
gridName,
realization,
propertyName,
isoDateOrInterval,
m_connector.networkAccessManager() ) );
}
RiaSumoConnector::waitForRepliesToFinish( blobIdReplies );
std::vector<QString> blobIds;
for ( auto reply : blobIdReplies )
{
blobIds.push_back( blobIdFromReply( reply, propertyName ) );
}
// Phase 2: download all resolved blobs as one group.
std::vector<QString> blobIdsToDownload;
for ( const auto& blobId : blobIds )
{
if ( !blobId.isEmpty() ) blobIdsToDownload.push_back( blobId );
}
const auto contentsByBlobId = m_connector.downloadBlobs( baseUrl, blobIdsToDownload );
// Anything missing failed to download, and is left out so the caller can fall back to a single request.
std::map<QString, QByteArray> contentsByTimestamp;
for ( size_t i = 0; i < blobIds.size(); i++ )
{
if ( blobIds[i].isEmpty() ) continue;
if ( auto it = contentsByBlobId.find( blobIds[i] ); it != contentsByBlobId.end() )
{
contentsByTimestamp[timestampsToFetch[i]] = it->second;
}
}
return contentsByTimestamp;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
QString RiaSumoGrid::propertyBlobIdPath( const SumoCaseId& caseId,
const QString& ensembleName,
const QString& gridName,
int realization,
const QString& propertyName,
const QString& isoDateOrInterval )
{
const QString encodedEnsembleName = QUrl::toPercentEncoding( ensembleName );
const QString encodedGridName = QUrl::toPercentEncoding( gridName );
const QString encodedPropertyName = QUrl::toPercentEncoding( propertyName );
QString path = QString( "/cases/%1/ensembles/%2/grids/%3/realizations/%4/properties/%5/blob_id" )
.arg( caseId.get() )
.arg( encodedEnsembleName )
.arg( encodedGridName )
.arg( realization )
.arg( encodedPropertyName );
// The timestamp/interval is an optional query parameter; omit it for static properties.
if ( !isoDateOrInterval.isEmpty() )
{
path += QString( "?property_iso_date_or_interval=%1" ).arg( QString( QUrl::toPercentEncoding( isoDateOrInterval ) ) );
}
return path;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
QString RiaSumoGrid::propertyBlobId( const SumoCaseId& caseId,
const QString& ensembleName,
const QString& gridName,
int realization,
const QString& propertyName,
const QString& isoDateOrInterval )
{
const QString path = propertyBlobIdPath( caseId, ensembleName, gridName, realization, propertyName, isoDateOrInterval );
return blobIdFromBody( m_connector.getBlocking( path ), "grid property", propertyName );
}
//--------------------------------------------------------------------------------------------------
/// Issue the blob id request for one grid property time step. The reply is returned unfinished, so the
/// caller decides how to wait for it: one at a time, or several at once when prefetching. networkManager
/// picks the connection pool, see the header comment.
//--------------------------------------------------------------------------------------------------
QNetworkReply* RiaSumoGrid::makePropertyBlobIdRequest( const QString& baseUrl,
const SumoCaseId& caseId,
const QString& ensembleName,
const QString& gridName,
int realization,
const QString& propertyName,
const QString& isoDateOrInterval,
QNetworkAccessManager* networkManager,
const void* cancelGroup )
{
const QString url = baseUrl + propertyBlobIdPath( caseId, ensembleName, gridName, realization, propertyName, isoDateOrInterval );
QNetworkRequest networkRequest;
networkRequest.setUrl( QUrl( url ) );
m_connector.addStandardHeader( networkRequest, m_connector.transferToken(), RiaCloudDefines::contentTypeJson() );
return m_connector.getAndTrackReply( networkManager, networkRequest, cancelGroup );
}
//--------------------------------------------------------------------------------------------------
/// Read the blob id off a finished blob id reply. The reply is consumed and scheduled for deletion.
///
/// Waiting for one specific reply, rather than for a signal shared by all blob id requests, is what makes
/// the mapping correct: a still-pending reply from an earlier property's request can no longer satisfy
/// this wait and hand us its blob id, which previously caused e.g. SWAT to be served the SWCR blob.
//--------------------------------------------------------------------------------------------------
QString RiaSumoGrid::blobIdFromReply( QNetworkReply* reply, const QString& propertyName )
{
if ( !reply ) return {};
const bool failed = !reply->isFinished() || reply->error() != QNetworkReply::NoError;
QByteArray body = failed ? QByteArray() : reply->readAll();
if ( failed )
{
RiaLogging::error( std::format( "Request grid property blob ID failed: '{}'", reply->errorString().toStdString() ) );
}
reply->deleteLater();
return blobIdFromBody( body, "grid property", propertyName );
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
QString RiaSumoGrid::blobIdFromBody( const QByteArray& body, const QString& kind, const QString& name )
{
const QString blobId = RiaSumoConnector::blobIdFromBody( body );
if ( blobId.isEmpty() ) return {};
RiaLogging::debug( std::format( "Received blob ID for {} '{}': {}", kind.toStdString(), name.toStdString(), blobId.toStdString() ) );
return blobId;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
std::vector<QString> RiaSumoGrid::parseGridNames( const QByteArray& body )
{
std::vector<QString> gridNames;
QJsonDocument doc = QJsonDocument::fromJson( body );
QJsonArray jsonArray = doc.array();
for ( const QJsonValue& value : jsonArray )
{
QString gridName = value.toString();
gridNames.push_back( gridName );
}
RiaLogging::debug( std::format( "Grid names count : {}", gridNames.size() ) );
return gridNames;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
SumoGridInfo RiaSumoGrid::parseGridInfo( const QByteArray& body )
{
SumoGridInfo gridInfo;
QJsonDocument doc = QJsonDocument::fromJson( body );
QJsonArray jsonArray = doc.array();
for ( const QJsonValue& value : jsonArray )
{
const QJsonObject realizationObj = value.toObject();
// A missing or null dimensions object leaves the counts at zero, which isValid() rejects, so a
// realization Sumo has no dimensions for falls back to a grid of its own.
const QJsonObject dimensionsObj = realizationObj["dimensions"].toObject();
SumoGridRealizationInfo realizationInfo;
realizationInfo.realization = realizationObj["realization"].toInt( -1 );
realizationInfo.dimensions.iCount = dimensionsObj["iCount"].toInt();
realizationInfo.dimensions.jCount = dimensionsObj["jCount"].toInt();
realizationInfo.dimensions.kCount = dimensionsObj["kCount"].toInt();
gridInfo.realizationInfos.push_back( realizationInfo );
}
return gridInfo;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
std::vector<SumoGridPropertyInfo> RiaSumoGrid::parsePropertyInfo( const QByteArray& body )
{
std::vector<SumoGridPropertyInfo> propertyInfos;
QJsonDocument doc = QJsonDocument::fromJson( body );
QJsonArray jsonArray = doc.array();
for ( const QJsonValue& value : jsonArray )
{
QJsonObject propertyObj = value.toObject();
SumoGridPropertyInfo propertyInfo;
propertyInfo.name = propertyObj["propertyName"].toString();
// isoDateOrInterval is null for static properties.
const auto isoValue = propertyObj["isoDateOrInterval"];
if ( !isoValue.isNull() ) propertyInfo.isoDateOrInterval = isoValue.toString();
propertyInfos.push_back( propertyInfo );
}
RiaLogging::debug( std::format( "Grid property info count : {}", propertyInfos.size() ) );
return propertyInfos;
}
@@ -0,0 +1,185 @@
/////////////////////////////////////////////////////////////////////////////////
//
// Copyright (C) 2024- Equinor ASA
//
// ResInsight is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// ResInsight is distributed in the hope that it will be useful, but WITHOUT ANY
// WARRANTY; without even the implied warranty of MERCHANTABILITY or
// FITNESS FOR A PARTICULAR PURPOSE.
//
// See the GNU General Public License at <http://www.gnu.org/licenses/gpl.html>
// for more details.
//
/////////////////////////////////////////////////////////////////////////////////
#pragma once
#include "RiaSumoDefines.h"
#include <QByteArray>
#include <QString>
#include <functional>
#include <map>
#include <vector>
class RiaSumoConnector;
class QNetworkReply;
class QNetworkAccessManager;
struct SumoGridDimensions
{
int iCount = 0;
int jCount = 0;
int kCount = 0;
bool isValid() const { return iCount > 0 && jCount > 0 && kCount > 0; }
bool operator==( const SumoGridDimensions& ) const = default;
};
struct SumoGridRealizationInfo
{
int realization = -1;
SumoGridDimensions dimensions;
};
//==================================================================================================
/// The realizations one grid of an ensemble exists for, each with the IJK dimensions Sumo reports. The
/// grid is identified by the request, so its name is not repeated here.
//==================================================================================================
struct SumoGridInfo
{
std::vector<SumoGridRealizationInfo> realizationInfos;
// The realizations this grid is available for.
std::vector<int> realizationIds() const;
// True when all the given realizations report the same valid IJK dimensions, which is what makes it
// safe to share one grid between them. False when any of them is missing from this grid or reports
// invalid dimensions, so the caller falls back to loading a grid per realization.
bool hasIdenticalDimensions( const std::vector<int>& realizations ) const;
};
struct SumoGridPropertyInfo
{
QString name;
// Empty for a static property. For a dynamic property this is either a single timestamp ("2018-01-01")
// or an interval ("2018-01-01/2019-01-01"). ResInsight currently only supports the single-timestamp form.
QString isoDateOrInterval;
};
//==================================================================================================
/// The grid data of a Sumo case: the grids of an ensemble, the grid geometry itself, and the grid
/// properties. Requests are made through RiaSumoConnector, which owns the connection and does the
/// transfers, and every call returns its result rather than leaving it in shared state.
//==================================================================================================
class RiaSumoGrid
{
public:
explicit RiaSumoGrid( RiaSumoConnector& connector );
// The names of the grids available in an ensemble.
std::vector<QString> gridNames( const SumoCaseId& caseId, const QString& ensembleName );
// The realizations one grid exists for, each with the IJK dimensions Sumo reports for it. Used by the
// reservoir grid ensemble, which needs the dimensions to decide whether the realizations can share a grid.
SumoGridInfo gridInfo( const SumoCaseId& caseId, const QString& ensembleName, const QString& gridName );
// The grid geometry as a binary roff blob.
QByteArray gridData( const SumoCaseId& caseId, const QString& ensembleName, const QString& gridName, int realization );
std::vector<SumoGridPropertyInfo>
propertyInfo( const SumoCaseId& caseId, const QString& ensembleName, const QString& gridName, int realization );
// One time step of one grid property, as a binary roff blob.
QByteArray propertyData( const SumoCaseId& caseId,
const QString& ensembleName,
const QString& gridName,
int realization,
const QString& propertyName,
const QString& isoDateOrInterval );
// Several time steps of one property, downloaded as one concurrent batch. The contents are keyed by
// the time step they belong to, and a time step that failed to download is absent from the result.
//
// Nothing is retained here: the caller decodes the contents and keeps the values, see
// RifReaderSumoGridProperty.
std::map<QString, QByteArray> propertyDataBatch( const SumoCaseId& caseId,
const QString& ensembleName,
const QString& gridName,
int realization,
const QString& propertyName,
const std::vector<QString>& isoDatesOrIntervals );
// The async twin of propertyDataBatch. All time steps are requested at once and onTimeStepReady is called
// for each as it arrives, on the connector thread, exactly once per requested step. Empty contents mean
// that step failed. Transfers get a generous deadline, see
// RiaSumoDefines::gridPropertyTransferTimeoutMillis.
//
// cancelGroup is passed on to RiaSumoConnector::downloadBlobAsync, see its documentation.
void propertyDataBatchAsync( const SumoCaseId& caseId,
const QString& ensembleName,
const QString& gridName,
int realization,
const QString& propertyName,
const std::vector<QString>& isoDatesOrIntervals,
const std::function<void( const QString&, const QByteArray& )>& onTimeStepReady,
const void* cancelGroup = nullptr );
private:
QString gridBlobId( const SumoCaseId& caseId, const QString& ensembleName, const QString& gridName, int realization );
QString propertyBlobId( const SumoCaseId& caseId,
const QString& ensembleName,
const QString& gridName,
int realization,
const QString& propertyName,
const QString& isoDateOrInterval );
// Runs on the transfer thread, so the base URL is resolved by the caller and passed in. networkManager
// lets the caller choose the connection pool: the shared one for a blocking fetch waited on directly, or
// the background one for a prefetch batch, see RiaSumoConnector::backgroundNetworkAccessManager.
// cancelGroup is passed on to RiaSumoConnector::getAndTrackReply, see its documentation; defaults to
// untracked for the batch path, which has no owner to cancel it early.
QNetworkReply* makePropertyBlobIdRequest( const QString& baseUrl,
const SumoCaseId& caseId,
const QString& ensembleName,
const QString& gridName,
int realization,
const QString& propertyName,
const QString& isoDateOrInterval,
QNetworkAccessManager* networkManager,
const void* cancelGroup = nullptr );
std::map<QString, QByteArray> fetchPropertyBatch( const QString& baseUrl,
const SumoCaseId& caseId,
const QString& ensembleName,
const QString& gridName,
int realization,
const QString& propertyName,
const std::vector<QString>& timestampsToFetch );
static QString propertyBlobIdPath( const SumoCaseId& caseId,
const QString& ensembleName,
const QString& gridName,
int realization,
const QString& propertyName,
const QString& isoDateOrInterval );
// 'kind' names what the blob holds, e.g. "grid" or "grid property", for the debug log.
static QString blobIdFromBody( const QByteArray& body, const QString& kind, const QString& name );
static QString blobIdFromReply( QNetworkReply* reply, const QString& propertyName );
static std::vector<QString> parseGridNames( const QByteArray& body );
static SumoGridInfo parseGridInfo( const QByteArray& body );
static std::vector<SumoGridPropertyInfo> parsePropertyInfo( const QByteArray& body );
private:
RiaSumoConnector& m_connector;
};
@@ -46,10 +46,9 @@ std::vector<QString> RiaSumoSummary::vectorNames( const SumoCaseId& caseId, cons
{
const QString encodedEnsembleName = QUrl::toPercentEncoding( ensembleName );
const QString url =
QString( "%1/cases/%2/ensembles/%3/vector_list" ).arg( m_connector.server() ).arg( caseId.get() ).arg( encodedEnsembleName );
const QString path = QString( "/cases/%1/ensembles/%2/vector_list" ).arg( caseId.get() ).arg( encodedEnsembleName );
return parseVectorNames( m_connector.getBlocking( url ) );
return parseVectorNames( m_connector.getBlocking( path ) );
}
//--------------------------------------------------------------------------------------------------
@@ -90,6 +89,13 @@ std::map<QString, QByteArray>
if ( namesToFetch.empty() ) return contentsByVectorName;
// The requests below are issued directly on the transfer thread, so the token and the address of the
// service have to be in place before the work is handed over.
m_connector.requestTokenBlocking();
const QString baseUrl = m_connector.serviceBaseUrl();
if ( baseUrl.isEmpty() ) return contentsByVectorName;
m_connector.runOnTransferThreadBlocking(
[&]()
{
@@ -97,7 +103,8 @@ std::map<QString, QByteArray>
std::vector<QNetworkReply*> blobIdReplies;
for ( const auto& vectorName : namesToFetch )
{
blobIdReplies.push_back( makeVectorBlobIdRequest( caseId, ensembleName, vectorName ) );
blobIdReplies.push_back(
makeVectorBlobIdRequest( baseUrl, caseId, ensembleName, vectorName, m_connector.networkAccessManager() ) );
}
RiaSumoConnector::waitForRepliesToFinish( blobIdReplies );
@@ -115,7 +122,7 @@ std::map<QString, QByteArray>
if ( !blobId.isEmpty() ) blobIdsToDownload.push_back( blobId );
}
const auto contentsByBlobId = m_connector.downloadBlobs( blobIdsToDownload );
const auto contentsByBlobId = m_connector.downloadBlobs( baseUrl, blobIdsToDownload );
// Anything missing failed; the caller falls back to fetching it on its own later.
for ( size_t i = 0; i < blobIds.size(); i++ )
@@ -144,12 +151,16 @@ std::map<QString, QByteArray>
void RiaSumoSummary::vectorDataAsync( const SumoCaseId& caseId,
const QString& ensembleName,
const std::vector<QString>& vectorNames,
const std::function<void( const QString&, const QByteArray& )>& onVectorReady )
const std::function<void( const QString&, const QByteArray& )>& onVectorReady,
const void* cancelGroup )
{
if ( vectorNames.empty() || !onVectorReady ) return;
const QString baseUrl = m_connector.serviceBaseUrl();
if ( baseUrl.isEmpty() ) return;
m_connector.runOnTransferThread(
[this, caseId, ensembleName, vectorNames, onVectorReady]()
[this, baseUrl, caseId, ensembleName, vectorNames, onVectorReady, cancelGroup]()
{
for ( const auto& vectorName : vectorNames )
{
@@ -158,7 +169,8 @@ void RiaSumoSummary::vectorDataAsync( const SumoCaseId&
m_connector.invokeOnConnectorThread( [onVectorReady, vectorName, contents]() { onVectorReady( vectorName, contents ); } );
};
auto blobIdReply = makeVectorBlobIdRequest( caseId, ensembleName, vectorName );
auto blobIdReply =
makeVectorBlobIdRequest( baseUrl, caseId, ensembleName, vectorName, m_connector.backgroundNetworkAccessManager(), cancelGroup );
if ( !blobIdReply )
{
deliver( {} );
@@ -172,7 +184,7 @@ void RiaSumoSummary::vectorDataAsync( const SumoCaseId&
QObject::connect( blobIdReply,
&QNetworkReply::finished,
blobIdReply,
[this, blobIdReply, vectorName, deliver]()
[this, blobIdReply, vectorName, deliver, cancelGroup]()
{
const QString blobId = blobIdFromReply( blobIdReply, vectorName );
if ( blobId.isEmpty() )
@@ -181,7 +193,7 @@ void RiaSumoSummary::vectorDataAsync( const SumoCaseId&
return;
}
m_connector.downloadBlobAsync( blobId, deliver );
m_connector.downloadBlobAsync( blobId, deliver, RiaSumoDefines::requestTimeoutMillis(), cancelGroup );
} );
}
} );
@@ -195,7 +207,7 @@ QByteArray RiaSumoSummary::parameterData( const SumoCaseId& caseId, const QStrin
const QString blobId = parameterBlobId( caseId, ensembleName );
if ( blobId.isEmpty() ) return {};
return m_connector.downloadBlobBlocking( blobId );
return m_connector.downloadBlobBlocking( blobId, "ensemble parameters" );
}
//--------------------------------------------------------------------------------------------------
@@ -203,37 +215,38 @@ QByteArray RiaSumoSummary::parameterData( const SumoCaseId& caseId, const QStrin
//--------------------------------------------------------------------------------------------------
QString RiaSumoSummary::vectorBlobId( const SumoCaseId& caseId, const QString& ensembleName, const QString& vectorName )
{
const QString url = vectorBlobIdUrl( caseId, ensembleName, vectorName );
const QString path = vectorBlobIdPath( caseId, ensembleName, vectorName );
return logBlobId( RiaSumoConnector::blobIdFromBody( m_connector.getBlocking( url ) ), vectorName );
return logBlobId( RiaSumoConnector::blobIdFromBody( m_connector.getBlocking( path ) ), vectorName );
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
QString RiaSumoSummary::vectorBlobIdUrl( const SumoCaseId& caseId, const QString& ensembleName, const QString& vectorName ) const
QString RiaSumoSummary::vectorBlobIdPath( const SumoCaseId& caseId, const QString& ensembleName, const QString& vectorName )
{
const QString encodedEnsembleName = QUrl::toPercentEncoding( ensembleName );
const QString encodedVectorName = QUrl::toPercentEncoding( vectorName );
return QString( "%1/cases/%2/ensembles/%3/vectors/%4/blob_id" )
.arg( m_connector.server() )
.arg( caseId.get() )
.arg( encodedEnsembleName )
.arg( encodedVectorName );
return QString( "/cases/%1/ensembles/%2/vectors/%3/blob_id" ).arg( caseId.get() ).arg( encodedEnsembleName ).arg( encodedVectorName );
}
//--------------------------------------------------------------------------------------------------
/// Issue the blob id request for one vector. The reply is returned unfinished, so the caller decides how
/// to wait for it: one at a time, or several at once when batching.
//--------------------------------------------------------------------------------------------------
QNetworkReply* RiaSumoSummary::makeVectorBlobIdRequest( const SumoCaseId& caseId, const QString& ensembleName, const QString& vectorName )
QNetworkReply* RiaSumoSummary::makeVectorBlobIdRequest( const QString& baseUrl,
const SumoCaseId& caseId,
const QString& ensembleName,
const QString& vectorName,
QNetworkAccessManager* networkManager,
const void* cancelGroup )
{
QNetworkRequest networkRequest;
networkRequest.setUrl( QUrl( vectorBlobIdUrl( caseId, ensembleName, vectorName ) ) );
networkRequest.setUrl( QUrl( baseUrl + vectorBlobIdPath( caseId, ensembleName, vectorName ) ) );
m_connector.addStandardHeader( networkRequest, m_connector.transferToken(), RiaCloudDefines::contentTypeJson() );
return m_connector.networkAccessManager()->get( networkRequest );
return m_connector.getAndTrackReply( networkManager, networkRequest, cancelGroup );
}
//--------------------------------------------------------------------------------------------------
@@ -273,24 +286,28 @@ QString RiaSumoSummary::logBlobId( const QString& blobId, const QString& vectorN
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
QString RiaSumoSummary::parameterBlobIdUrl( const SumoCaseId& caseId, const QString& ensembleName ) const
QString RiaSumoSummary::parameterBlobIdPath( const SumoCaseId& caseId, const QString& ensembleName )
{
const QString encodedEnsembleName = QUrl::toPercentEncoding( ensembleName );
return QString( "%1/cases/%2/ensembles/%3/parameters/blob_id" ).arg( m_connector.server() ).arg( caseId.get() ).arg( encodedEnsembleName );
return QString( "/cases/%1/ensembles/%2/parameters/blob_id" ).arg( caseId.get() ).arg( encodedEnsembleName );
}
//--------------------------------------------------------------------------------------------------
/// Issue the blob id request for the ensemble parameters. The reply is returned unfinished, so the caller
/// decides how to wait for it.
//--------------------------------------------------------------------------------------------------
QNetworkReply* RiaSumoSummary::makeParameterBlobIdRequest( const SumoCaseId& caseId, const QString& ensembleName )
QNetworkReply* RiaSumoSummary::makeParameterBlobIdRequest( const QString& baseUrl,
const SumoCaseId& caseId,
const QString& ensembleName,
QNetworkAccessManager* networkManager,
const void* cancelGroup )
{
QNetworkRequest networkRequest;
networkRequest.setUrl( QUrl( parameterBlobIdUrl( caseId, ensembleName ) ) );
networkRequest.setUrl( QUrl( baseUrl + parameterBlobIdPath( caseId, ensembleName ) ) );
m_connector.addStandardHeader( networkRequest, m_connector.transferToken(), RiaCloudDefines::contentTypeJson() );
return m_connector.networkAccessManager()->get( networkRequest );
return m_connector.getAndTrackReply( networkManager, networkRequest, cancelGroup );
}
//--------------------------------------------------------------------------------------------------
@@ -299,17 +316,22 @@ QNetworkReply* RiaSumoSummary::makeParameterBlobIdRequest( const SumoCaseId& cas
//--------------------------------------------------------------------------------------------------
void RiaSumoSummary::parameterDataAsync( const SumoCaseId& caseId,
const QString& ensembleName,
const std::function<void( const QByteArray& )>& onParametersReady )
const std::function<void( const QByteArray& )>& onParametersReady,
const void* cancelGroup )
{
if ( !onParametersReady ) return;
const QString baseUrl = m_connector.serviceBaseUrl();
if ( baseUrl.isEmpty() ) return;
m_connector.runOnTransferThread(
[this, caseId, ensembleName, onParametersReady]()
[this, baseUrl, caseId, ensembleName, onParametersReady, cancelGroup]()
{
auto deliver = [this, onParametersReady]( const QByteArray& contents )
{ m_connector.invokeOnConnectorThread( [onParametersReady, contents]() { onParametersReady( contents ); } ); };
auto blobIdReply = makeParameterBlobIdRequest( caseId, ensembleName );
auto blobIdReply =
makeParameterBlobIdRequest( baseUrl, caseId, ensembleName, m_connector.backgroundNetworkAccessManager(), cancelGroup );
if ( !blobIdReply )
{
deliver( {} );
@@ -321,7 +343,7 @@ void RiaSumoSummary::parameterDataAsync( const SumoCaseId&
QObject::connect( blobIdReply,
&QNetworkReply::finished,
blobIdReply,
[this, blobIdReply, deliver]()
[this, blobIdReply, deliver, cancelGroup]()
{
const QString blobId = blobIdFromReply( blobIdReply, "parameters" );
if ( blobId.isEmpty() )
@@ -330,7 +352,7 @@ void RiaSumoSummary::parameterDataAsync( const SumoCaseId&
return;
}
m_connector.downloadBlobAsync( blobId, deliver );
m_connector.downloadBlobAsync( blobId, deliver, RiaSumoDefines::requestTimeoutMillis(), cancelGroup );
} );
} );
}
@@ -340,7 +362,7 @@ void RiaSumoSummary::parameterDataAsync( const SumoCaseId&
//--------------------------------------------------------------------------------------------------
QString RiaSumoSummary::parameterBlobId( const SumoCaseId& caseId, const QString& ensembleName )
{
const QString blobId = RiaSumoConnector::blobIdFromBody( m_connector.getBlocking( parameterBlobIdUrl( caseId, ensembleName ) ) );
const QString blobId = RiaSumoConnector::blobIdFromBody( m_connector.getBlocking( parameterBlobIdPath( caseId, ensembleName ) ) );
if ( !blobId.isEmpty() )
{
@@ -29,6 +29,7 @@
class RiaSumoConnector;
class QNetworkReply;
class QNetworkAccessManager;
//==================================================================================================
/// The summary data of a Sumo case: the vectors an ensemble has, their values, and the ensemble
@@ -53,10 +54,13 @@ public:
// The same again, but without waiting: all vectors are requested at once and onVectorReady is called for
// each as it arrives, on the thread the connector lives on. Empty contents mean that vector failed, and
// the callback is called exactly once per requested vector.
//
// cancelGroup is passed on to RiaSumoConnector::downloadBlobAsync, see its documentation.
void vectorDataAsync( const SumoCaseId& caseId,
const QString& ensembleName,
const std::vector<QString>& vectorNames,
const std::function<void( const QString&, const QByteArray& )>& onVectorReady );
const std::function<void( const QString&, const QByteArray& )>& onVectorReady,
const void* cancelGroup = nullptr );
// The ensemble parameters, as a parquet blob.
QByteArray parameterData( const SumoCaseId& caseId, const QString& ensembleName );
@@ -65,16 +69,32 @@ public:
// the first request for them can take a while and is not something to hold the user interface for.
void parameterDataAsync( const SumoCaseId& caseId,
const QString& ensembleName,
const std::function<void( const QByteArray& )>& onParametersReady );
const std::function<void( const QByteArray& )>& onParametersReady,
const void* cancelGroup = nullptr );
QString vectorBlobId( const SumoCaseId& caseId, const QString& ensembleName, const QString& vectorName );
QString parameterBlobId( const SumoCaseId& caseId, const QString& ensembleName );
private:
QString vectorBlobIdUrl( const SumoCaseId& caseId, const QString& ensembleName, const QString& vectorName ) const;
QString parameterBlobIdUrl( const SumoCaseId& caseId, const QString& ensembleName ) const;
QNetworkReply* makeParameterBlobIdRequest( const SumoCaseId& caseId, const QString& ensembleName );
QNetworkReply* makeVectorBlobIdRequest( const SumoCaseId& caseId, const QString& ensembleName, const QString& vectorName );
static QString vectorBlobIdPath( const SumoCaseId& caseId, const QString& ensembleName, const QString& vectorName );
static QString parameterBlobIdPath( const SumoCaseId& caseId, const QString& ensembleName );
// Run on the transfer thread, so the base URL is resolved by the caller and passed in. networkManager
// lets the caller choose the connection pool: the shared one for a blocking fetch waited on directly, or
// the background one for a prefetch batch, see RiaSumoConnector::backgroundNetworkAccessManager.
// cancelGroup is passed on to RiaSumoConnector::getAndTrackReply, see its documentation; defaults to
// untracked for the batch path, which has no owner to cancel it early.
QNetworkReply* makeParameterBlobIdRequest( const QString& baseUrl,
const SumoCaseId& caseId,
const QString& ensembleName,
QNetworkAccessManager* networkManager,
const void* cancelGroup = nullptr );
QNetworkReply* makeVectorBlobIdRequest( const QString& baseUrl,
const SumoCaseId& caseId,
const QString& ensembleName,
const QString& vectorName,
QNetworkAccessManager* networkManager,
const void* cancelGroup = nullptr );
static QString blobIdFromReply( QNetworkReply* reply, const QString& vectorName );
static QString logBlobId( const QString& blobId, const QString& vectorName );
@@ -177,12 +177,13 @@ QString RiaProjectFileTools::transferPathsToGlobalPathList( RimProject* project
{
if ( gridCase->displayNameType() == RimCaseDisplayNameTools::DisplayName::CUSTOM )
{
// At this point, after the replace of variables into caf::FilePath objects, the variable name is
// stored in the summary case object. Read out the variable name and append "_name" for custom
// summary variables.
// The grid file path was replaced by a path variable above, so reuse that variable as the key for
// the case name. A case not written to the project file still holds its raw path, and
// RiaVariableMapper cannot read back a key holding '/', '-' or '.', so leave that name alone.
QString variableName = gridCase->gridFileName();
variableName = variableName.remove( RiaVariableMapper::variableToken() );
if ( !variableName.startsWith( RiaVariableMapper::variableToken() ) ) continue;
variableName = variableName.remove( RiaVariableMapper::variableToken() );
variableName = RiaVariableMapper::variableToken() + variableName + RiaVariableMapper::postfixName() +
RiaVariableMapper::variableToken();
+1 -1
View File
@@ -110,7 +110,7 @@ list(
ProjectDataModel/GridCrossPlots/CellFilters/CMakeLists_files.cmake
ProjectDataModel/Jobs/CMakeLists_files.cmake
ProjectDataModel/Summary/CMakeLists_files.cmake
ProjectDataModel/Summary/Sumo/CMakeLists_files.cmake
ProjectDataModel/Summary/Cloud/CMakeLists_files.cmake
ProjectDataModel/Summary/Ensemble/CMakeLists_files.cmake
ProjectDataModel/Flow/CMakeLists_files.cmake
ProjectDataModel/Formations/CMakeLists_files.cmake
@@ -94,11 +94,14 @@ void RicCreateReservoirGridEnsembleFromFileSetFeature::onActionTriggered( bool i
RimReservoirGridEnsemble* gridEnsemble = new RimReservoirGridEnsemble();
gridEnsemble->setEnsembleFileSet( fileSet );
gridEnsemble->createGridCasesFromEnsembleFileSet();
// Put the ensemble in the project tree before the cases are created, so RimProject::assignCaseIdToCase
// finds them as descendants of the project and hands out unique case ids.
project->assignIdToCaseGroup( gridEnsemble );
eclipseCaseColl->reservoirGridEnsembles.push_back( gridEnsemble );
gridEnsemble->createGridCasesFromEnsembleFileSet();
gridEnsemble->loadDataAndUpdate();
// Create view for first case if available
@@ -68,6 +68,7 @@ void RicNewStatisticsContourMapFeature::addStatisticsContourMap( RimReservoirGri
}
statisticsContourMap->setEclipseCase( cases[0] );
statisticsContourMap->switchToSelectedSourceCase();
ensemble->addStatisticsContourMap( statisticsContourMap );
if ( auto* pdmObject = dynamic_cast<caf::PdmObject*>( ensemble ) ) pdmObject->updateConnectedEditors();
@@ -23,13 +23,15 @@
#include "ContourMap/RimEclipseContourMapView.h"
#include "Rim3dView.h"
#include "RimEclipseCase.h"
#include "RimEclipseCaseEnsemble.h"
#include "RimEclipseView.h"
#include "RimEclipseViewCollection.h"
#include "RimGeoMechCase.h"
#include "RimGeoMechView.h"
#include "RimReservoirGridEnsembleBase.h"
#include "Riu3DMainWindowTools.h"
#include "cafPdmObject.h"
#include "cafSelectionManager.h"
#include <QAction>
@@ -39,13 +41,20 @@ CAF_CMD_SOURCE_INIT( RicNewViewForGridEnsembleFeature, "RicNewViewForGridEnsembl
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RicNewViewForGridEnsembleFeature::addView( RimEclipseCaseEnsemble* eclipseCaseEnsemble )
void RicNewViewForGridEnsembleFeature::addView( RimReservoirGridEnsembleBase* gridEnsemble )
{
std::vector<RimEclipseCase*> cases = eclipseCaseEnsemble->cases();
if ( !gridEnsemble ) return;
auto viewCollection = gridEnsemble->viewCollection();
if ( !viewCollection ) return;
std::vector<RimEclipseCase*> cases = gridEnsemble->sourceCases();
if ( cases.empty() ) return;
auto newView = eclipseCaseEnsemble->addViewForCase( cases[0] );
eclipseCaseEnsemble->updateConnectedEditors();
auto newView = viewCollection->addView( cases[0] );
// The ensemble types are not PdmObject in the shared base, so get there by cast.
if ( auto pdmObject = dynamic_cast<caf::PdmObject*>( gridEnsemble ) ) pdmObject->updateConnectedEditors();
Riu3DMainWindowTools::setExpanded( newView );
@@ -58,7 +67,7 @@ void RicNewViewForGridEnsembleFeature::addView( RimEclipseCaseEnsemble* eclipseC
//--------------------------------------------------------------------------------------------------
bool RicNewViewForGridEnsembleFeature::isCommandEnabled() const
{
return selectedEclipseCaseEnsemble() != nullptr;
return selectedGridEnsemble() != nullptr;
}
//--------------------------------------------------------------------------------------------------
@@ -66,8 +75,7 @@ bool RicNewViewForGridEnsembleFeature::isCommandEnabled() const
//--------------------------------------------------------------------------------------------------
void RicNewViewForGridEnsembleFeature::onActionTriggered( bool isChecked )
{
RimEclipseCaseEnsemble* eclipseCaseEnsemble = selectedEclipseCaseEnsemble();
addView( eclipseCaseEnsemble );
addView( selectedGridEnsemble() );
}
//--------------------------------------------------------------------------------------------------
@@ -82,7 +90,7 @@ void RicNewViewForGridEnsembleFeature::setupActionLook( QAction* actionToSetup )
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
RimEclipseCaseEnsemble* RicNewViewForGridEnsembleFeature::selectedEclipseCaseEnsemble()
RimReservoirGridEnsembleBase* RicNewViewForGridEnsembleFeature::selectedGridEnsemble()
{
return caf::SelectionManager::instance()->selectedItemOfType<RimEclipseCaseEnsemble>();
return caf::SelectionManager::instance()->selectedItemOfType<RimReservoirGridEnsembleBase>();
}
@@ -20,8 +20,8 @@
#include "cafCmdFeature.h"
class RimEclipseCaseEnsemble;
class RimEclipseView;
class RimReservoirGridEnsembleBase;
//==================================================================================================
///
@@ -31,12 +31,12 @@ class RicNewViewForGridEnsembleFeature : public caf::CmdFeature
CAF_CMD_HEADER_INIT;
public:
static void addView( RimEclipseCaseEnsemble* eclipseCaseEnsemble );
static void addView( RimReservoirGridEnsembleBase* gridEnsemble );
protected:
bool isCommandEnabled() const override;
void onActionTriggered( bool isChecked ) override;
void setupActionLook( QAction* actionToSetup ) override;
static RimEclipseCaseEnsemble* selectedEclipseCaseEnsemble();
static RimReservoirGridEnsembleBase* selectedGridEnsemble();
};
@@ -26,6 +26,7 @@
#include "RiuSummaryCurveDefinitionKeywords.h"
#include "Cloud/RimSummaryEnsembleSumo.h"
#include "RimDeltaSummaryEnsemble.h"
#include "RimEnsembleCurveSet.h"
#include "RimEnsembleCurveSetCollection.h"
@@ -44,7 +45,6 @@
#include "RimSummaryEnsemble.h"
#include "RimSummaryMultiPlot.h"
#include "RimSummaryPlot.h"
#include "Sumo/RimSummaryEnsembleSumo.h"
#include "RiuPlotMainWindow.h"
#include "RiuPlotMainWindowTools.h"
@@ -1,5 +1,6 @@
set(SOURCE_GROUP_SOURCE_FILES
${CMAKE_CURRENT_LIST_DIR}/RicCreateSumoEnsembleFeature.cpp
${CMAKE_CURRENT_LIST_DIR}/RicCreateSumoReservoirGridEnsembleFeature.cpp
${CMAKE_CURRENT_LIST_DIR}/RicDeleteSumoTokenFeature.cpp
)
@@ -19,9 +19,12 @@
#include "RicCreateSumoEnsembleFeature.h"
#include "RiaDefines.h"
#include "RiaGuiApplication.h"
#include "Cloud/RimCloudDataSourceCollection.h"
#include "Sumo/RimSumoDataSource.h"
#include "Cloud/RimSumoDataSource.h"
#include "RiuPlotMainWindow.h"
#include "cafSelectionManagerTools.h"
@@ -29,6 +32,16 @@
CAF_CMD_SOURCE_INIT( RicCreateSumoEnsembleFeature, "RicCreateSumoEnsembleFeature" );
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
bool RicCreateSumoEnsembleFeature::isCommandEnabled() const
{
// The context menu is raised from the window holding the tree, so the active window is the one that was
// right-clicked. Same discrimination as RicViewZoomAllFeature.
return dynamic_cast<RiuPlotMainWindow*>( RiaGuiApplication::activeWindow() ) != nullptr;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
@@ -28,6 +28,11 @@ class RicCreateSumoEnsembleFeature : public caf::CmdFeature
CAF_CMD_HEADER_INIT;
protected:
// A summary ensemble belongs to the plot window, so the command is only offered there. Returning false
// removes the entry from the context menu rather than disabling it, see
// caf::CmdFeatureMenuBuilder::appendToMenu.
bool isCommandEnabled() const override;
void onActionTriggered( bool isChecked ) override;
void setupActionLook( QAction* actionToSetup ) override;
};
@@ -0,0 +1,248 @@
/////////////////////////////////////////////////////////////////////////////////
//
// Copyright (C) 2026 Equinor ASA
//
// ResInsight is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// ResInsight is distributed in the hope that it will be useful, but WITHOUT ANY
// WARRANTY; without even the implied warranty of MERCHANTABILITY or
// FITNESS FOR A PARTICULAR PURPOSE.
//
// See the GNU General Public License at <http://www.gnu.org/licenses/gpl.html>
// for more details.
//
/////////////////////////////////////////////////////////////////////////////////
#include "RicCreateSumoReservoirGridEnsembleFeature.h"
#include "RiaApplication.h"
#include "RiaDefines.h"
#include "RiaGuiApplication.h"
#include "RiaLogging.h"
#include "RiaStdStringTools.h"
#include "Cloud/RiaSumoConnector.h"
#include "RicNewViewFeature.h"
#include "Cloud/RimSumoDataSource.h"
#include "Rim3dView.h"
#include "RimEclipseCase.h"
#include "RimEclipseCaseCollection.h"
#include "RimOilField.h"
#include "RimProject.h"
#include "RimReservoirGridEnsembleSumo.h"
#include "RimViewNameConfig.h"
#include "RiuMainWindow.h"
#include "cafSelectionManagerTools.h"
#include <QAction>
#include <algorithm>
#include <format>
CAF_CMD_SOURCE_INIT( RicCreateSumoReservoirGridEnsembleFeature, "RicCreateSumoReservoirGridEnsembleFeature" );
namespace
{
//--------------------------------------------------------------------------------------------------
/// Log the IJK dimensions Sumo reports for each selected realization, and the grid mode they lead to.
//--------------------------------------------------------------------------------------------------
void logRealizationDimensions( const QString& gridName, const SumoGridInfo& gridInfo, const std::vector<int>& realizations, bool dimensionsAreIdentical )
{
RiaLogging::debug( std::format( "Sumo grid '{}': {} of {} realizations selected",
gridName.toStdString(),
realizations.size(),
gridInfo.realizationInfos.size() ) );
for ( int realization : realizations )
{
auto it = std::ranges::find( gridInfo.realizationInfos, realization, &SumoGridRealizationInfo::realization );
if ( it == gridInfo.realizationInfos.end() ) continue;
RiaLogging::debug(
std::format( " realization {}: {}x{}x{}", realization, it->dimensions.iCount, it->dimensions.jCount, it->dimensions.kCount ) );
}
RiaLogging::debug( dimensionsAreIdentical ? " -> dimensions identical, shared grid mode" : " -> dimensions differ, individual grids" );
}
} // namespace
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RicCreateSumoReservoirGridEnsembleFeature::onActionTriggered( bool isChecked )
{
auto dataSources = caf::selectedObjectsByType<RimSumoDataSource*>();
for ( auto dataSource : dataSources )
{
createGridEnsemble( dataSource );
}
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RicCreateSumoReservoirGridEnsembleFeature::createGridEnsemble( RimSumoDataSource* dataSource )
{
if ( !dataSource ) return;
const QString gridName = dataSource->selectedGridName();
if ( gridName.isEmpty() )
{
RiaLogging::warning( "No grid selected. Unable to create grid ensemble from Sumo." );
return;
}
RimProject* project = RimProject::current();
if ( !project ) return;
RimOilField* oilfield = project->activeOilField();
if ( !oilfield ) return;
RimEclipseCaseCollection* analysisModels = oilfield->analysisModels();
if ( !analysisModels ) return;
auto sumoConnector = RiaApplication::instance()->makeSumoConnector();
if ( !sumoConnector )
{
RiaLogging::warning( "No Sumo connector available, unable to create grid ensemble from Sumo." );
return;
}
// The realizations of the selected grid, each with the IJK dimensions Sumo reports for it. The
// dimensions decide whether the realizations can share one grid, so no grid is downloaded to find out.
const auto gridInfo = sumoConnector->grid().gridInfo( dataSource->caseId(), dataSource->ensembleName(), gridName );
// The response body is discarded on any failed request, so an empty result is also what a failed
// transfer looks like. Report that separately from a grid the filter selected no realizations of.
if ( gridInfo.realizationInfos.empty() )
{
RiaLogging::warning( QString( "No realizations found for grid '%1' in ensemble '%2'. No grid ensemble created, "
"see the log for a failed request." )
.arg( gridName, dataSource->ensembleName() )
.toStdString() );
return;
}
const auto gridRealizations = gridInfo.realizationIds();
// The realizations selected on the data source are the realizations of the ensemble. That selection says
// nothing about which data types exist for them, so the grid can be missing for some. A realization the
// grid does not exist for would only produce a case failing to load, so skip it - but report it, so an
// ensemble holding fewer cases than the selection is visible rather than silent.
std::vector<int> realizations;
std::vector<int> realizationsWithoutGrid;
for ( const QString& realizationId : dataSource->selectedRealizationIds() )
{
bool ok = false;
int realization = realizationId.toInt( &ok );
if ( !ok ) continue;
if ( std::ranges::find( gridRealizations, realization ) == gridRealizations.end() )
{
realizationsWithoutGrid.push_back( realization );
continue;
}
realizations.push_back( realization );
}
if ( !realizationsWithoutGrid.empty() )
{
std::ranges::sort( realizationsWithoutGrid );
RiaLogging::warning( QString( "Grid '%1' has no data for %2 of the selected realizations, no cases created for "
"them: %3" )
.arg( gridName )
.arg( realizationsWithoutGrid.size() )
.arg( QString::fromStdString( RiaStdStringTools::formatRangeSelection( realizationsWithoutGrid ) ) )
.toStdString() );
}
// The realizations of the data source are the realizations of the ensemble, so the grid should not report
// anything outside them. Leave a trace if it does, rather than reporting an inconsistency the user cannot
// act on. realizations holds exactly the grid realizations that are also selected, so a grid realization
// missing from it is one the ensemble does not have.
std::vector<int> realizationsNotInEnsemble;
for ( int gridRealization : gridRealizations )
{
if ( std::ranges::find( realizations, gridRealization ) == realizations.end() )
{
realizationsNotInEnsemble.push_back( gridRealization );
}
}
if ( !realizationsNotInEnsemble.empty() )
{
RiaLogging::debug( QString( "Grid '%1' reports %2 realization(s) that are not selected realizations of the ensemble: %3" )
.arg( gridName )
.arg( realizationsNotInEnsemble.size() )
.arg( QString::fromStdString( RiaStdStringTools::formatRangeSelection( realizationsNotInEnsemble ) ) )
.toStdString() );
}
if ( realizations.empty() )
{
RiaLogging::warning( QString( "No selected realizations have the grid '%1'. No grid ensemble created." ).arg( gridName ).toStdString() );
return;
}
const bool dimensionsAreIdentical = gridInfo.hasIdenticalDimensions( realizations );
// Log the dimensions the endpoint reported, so a grid ensemble that does not share its grid can be
// told apart from an endpoint reporting the wrong dimensions, before anything is downloaded.
logRealizationDimensions( gridName, gridInfo, realizations, dimensionsAreIdentical );
auto* gridEnsemble = new RimReservoirGridEnsembleSumo;
gridEnsemble->setName( QString( "%1 - %2" ).arg( dataSource->ensembleName(), gridName ) );
gridEnsemble->setSumoSource( dataSource, gridName, realizations, dimensionsAreIdentical );
gridEnsemble->setGridRealizations( gridRealizations );
gridEnsemble->setDoComputeMobileVolumeWeightedMean( dataSource->doComputeMobileVolumeWeightedMean() );
// Put the ensemble in the project tree before the cases are created, so RimProject::assignCaseIdToCase
// finds them as descendants of the project and hands out unique case ids.
project->assignIdToCaseGroup( gridEnsemble );
analysisModels->reservoirGridEnsembles.push_back( gridEnsemble );
gridEnsemble->createGridCasesFromSumoSource();
gridEnsemble->loadDataAndUpdate();
analysisModels->updateAllRequiredEditors();
auto allCases = gridEnsemble->cases();
if ( allCases.empty() )
{
RiaLogging::warning( "No valid realizations selected. No grid ensemble created." );
return;
}
if ( auto view = RicNewViewFeature::addReservoirView( allCases.front(), nullptr, gridEnsemble->viewCollection() ) )
{
view->nameConfig()->setAddCaseName( true );
}
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
bool RicCreateSumoReservoirGridEnsembleFeature::isCommandEnabled() const
{
// The context menu is raised from the window holding the tree, so the active window is the one that was
// right-clicked. Same discrimination as RicViewZoomAllFeature.
return dynamic_cast<RiuMainWindow*>( RiaGuiApplication::activeWindow() ) != nullptr;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RicCreateSumoReservoirGridEnsembleFeature::setupActionLook( QAction* actionToSetup )
{
actionToSetup->setText( "Create Reservoir Grid Ensemble" + RiaDefines::betaFeaturePostfix() );
actionToSetup->setIcon( QIcon( ":/GridCaseGroup16x16.png" ) );
}
@@ -0,0 +1,43 @@
/////////////////////////////////////////////////////////////////////////////////
//
// Copyright (C) 2026 Equinor ASA
//
// ResInsight is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// ResInsight is distributed in the hope that it will be useful, but WITHOUT ANY
// WARRANTY; without even the implied warranty of MERCHANTABILITY or
// FITNESS FOR A PARTICULAR PURPOSE.
//
// See the GNU General Public License at <http://www.gnu.org/licenses/gpl.html>
// for more details.
//
/////////////////////////////////////////////////////////////////////////////////
#pragma once
#include "cafCmdFeature.h"
class RimSumoDataSource;
//==================================================================================================
/// Create a grid ensemble of the selected Sumo grid, as a RimReservoirGridEnsembleSumo.
//==================================================================================================
class RicCreateSumoReservoirGridEnsembleFeature : public caf::CmdFeature
{
CAF_CMD_HEADER_INIT;
protected:
// A grid ensemble belongs to the 3D window, so the command is only offered there. Returning false
// removes the entry from the context menu rather than disabling it, see
// caf::CmdFeatureMenuBuilder::appendToMenu.
bool isCommandEnabled() const override;
void onActionTriggered( bool isChecked ) override;
void setupActionLook( QAction* actionToSetup ) override;
private:
static void createGridEnsemble( RimSumoDataSource* dataSource );
};
@@ -2,6 +2,7 @@ set(SOURCE_GROUP_SOURCE_FILES
${CMAKE_CURRENT_LIST_DIR}/RifTextDataTableFormatter.cpp
${CMAKE_CURRENT_LIST_DIR}/RifEclipseInputFileTools.cpp
${CMAKE_CURRENT_LIST_DIR}/RifRoffFileTools.cpp
${CMAKE_CURRENT_LIST_DIR}/RifReaderSumoGridProperty.cpp
${CMAKE_CURRENT_LIST_DIR}/RifEclipseOutputFileTools.cpp
${CMAKE_CURRENT_LIST_DIR}/RifEclipseRestartFilesetAccess.cpp
${CMAKE_CURRENT_LIST_DIR}/RifEclipseUnifiedRestartFileAccess.cpp
@@ -0,0 +1,651 @@
/////////////////////////////////////////////////////////////////////////////////
//
// Copyright (C) 2026 Equinor ASA
//
// ResInsight is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// ResInsight is distributed in the hope that it will be useful, but WITHOUT ANY
// WARRANTY; without even the implied warranty of MERCHANTABILITY or
// FITNESS FOR A PARTICULAR PURPOSE.
//
// See the GNU General Public License at <http://www.gnu.org/licenses/gpl.html>
// for more details.
//
/////////////////////////////////////////////////////////////////////////////////
#include "RifReaderSumoGridProperty.h"
#include "RiaLogging.h"
#include "RiaRegressionTestRunner.h"
#include "Cloud/RiaSumoConnector.h"
#include "Cloud/RiaSumoDefines.h"
#include "RifRoffFileTools.h"
#include "RigCaseCellResultsData.h"
#include "RigEclipseCaseData.h"
#include "RigEclipseResultAddress.h"
#include "RigMainGrid.h"
#include <QEventLoop>
#include <QTimer>
#include <algorithm>
#include <cmath>
#include <sstream>
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
RifReaderSumoGridProperty::RifReaderSumoGridProperty( RiaSumoConnector* connector,
const QString& caseId,
const QString& ensembleName,
const QString& gridName,
int realization )
: m_connector( connector )
, m_caseId( caseId )
, m_ensembleName( ensembleName )
, m_gridName( gridName )
, m_realization( realization )
, m_caseData( nullptr )
, m_lifetimeToken( std::make_shared<bool>( true ) )
{
}
//--------------------------------------------------------------------------------------------------
/// Aborts any transfers this reader still has in flight, see RiaSumoConnector::cancelGroup.
///
/// cancelGroup() can make Qt deliver an aborted reply's finished() synchronously, re-entering this
/// reader's own completion lambdas while this destructor is still on the stack. Those lambdas bail
/// out once m_lifetimeToken has expired, so it must be invalidated *before* calling cancelGroup(),
/// not left to expire implicitly afterwards - otherwise a reentrant callback could still touch this
/// or m_caseData while both are mid-destruction.
//--------------------------------------------------------------------------------------------------
RifReaderSumoGridProperty::~RifReaderSumoGridProperty()
{
void* lifetimeTokenKey = m_lifetimeToken.get();
m_lifetimeToken.reset();
if ( m_connector ) m_connector->cancelGroup( lifetimeTokenKey );
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RifReaderSumoGridProperty::setPendingChangedCallback( PendingChangedCallback callback )
{
m_onPendingChanged = std::move( callback );
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RifReaderSumoGridProperty::setTimeStepArrivedCallback( TimeStepArrivedCallback callback )
{
m_onTimeStepArrived = std::move( callback );
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RifReaderSumoGridProperty::setBatchProgressChangedCallback( BatchProgressChangedCallback callback )
{
m_onBatchProgressChanged = std::move( callback );
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RifReaderSumoGridProperty::setStaticProperties( const std::vector<QString>& propertyNames )
{
m_staticProperties = propertyNames;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RifReaderSumoGridProperty::setDynamicProperties( const std::map<QString, std::vector<QString>>& propertyNameToTimestamps )
{
m_dynamicTimestamps = propertyNameToTimestamps;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
bool RifReaderSumoGridProperty::open( const QString& /*fileName*/, RigEclipseCaseData* eclipseCase )
{
// The grid geometry is loaded elsewhere; only keep the case data for cell count and active cell masking.
m_caseData = eclipseCase;
return true;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
bool RifReaderSumoGridProperty::staticResult( const QString& result, RiaDefines::PorosityModelType matrixOrFracture, std::vector<double>* values )
{
if ( matrixOrFracture != RiaDefines::PorosityModelType::MATRIX_MODEL ) return false;
// Only fetch properties this reader owns; other static results (e.g. computed DEPTH) are not on Sumo.
if ( std::find( m_staticProperties.begin(), m_staticProperties.end(), result ) == m_staticProperties.end() ) return false;
return fetchAndDecode( result, "", values );
}
//--------------------------------------------------------------------------------------------------
/// A displayed dynamic property is read one time step at a time. Fires a batch request for every missing
/// time step of this property alongside the one asked for, then blocks the calling thread - showing a
/// progress bar - until the requested step itself has arrived (or the transfer times out), so the caller
/// gets back real values instead of a placeholder, the same contract a disk-based reader already has.
//--------------------------------------------------------------------------------------------------
bool RifReaderSumoGridProperty::dynamicResult( const QString& result,
RiaDefines::PorosityModelType matrixOrFracture,
size_t stepIndex,
std::vector<double>* values )
{
if ( matrixOrFracture != RiaDefines::PorosityModelType::MATRIX_MODEL ) return false;
auto it = m_dynamicTimestamps.find( result );
if ( it == m_dynamicTimestamps.end() || stepIndex >= it->second.size() ) return false;
// The timestamp list is aligned with the case's common time steps. An empty entry means this property has
// no data at that time step, so report "no data" instead of fetching another step's values.
const std::vector<QString>& timestamps = it->second;
const QString& isoDateOrInterval = timestamps[stepIndex];
if ( isoDateOrInterval.isEmpty() ) return false;
// Skip straight to the result if this exact step is already resolved (loaded, or permanently failed) and
// nothing is in flight for it: requestTimeStepsAsync marks its primary stepIndex pending unconditionally,
// with no "already loaded" check of its own (only the other look-ahead steps get that check), so without
// this guard a later call for an already-resolved step - e.g. from a redraw after a sibling step arrives -
// would mark it pending again and re-fetch it from the network every time.
const bool stepAlreadyPending = m_pending.count( PendingKey{ result, stepIndex } ) > 0;
auto* preExistingSlot = resultValueSlot( result, stepIndex );
const bool stepAlreadyResolved = !stepAlreadyPending && preExistingSlot && !preExistingSlot->empty();
if ( !stepAlreadyResolved )
{
// Always try to fold in the rest of this property's missing time steps alongside the one asked for -
// requestTimeStepsAsync/timeStepsToFetch already skip whatever is already pending or loaded, so this
// is safe to call whenever this step itself is not resolved yet.
requestTimeStepsAsync( result, timestamps, timeStepsToFetch( result, timestamps, stepIndex ) );
}
// Block on the requested step: the caller (a view, calculation, contour map, etc.) is not designed to
// cope with partially-loaded Sumo data, so the simplest correct contract is to make this call behave
// like a disk-based reader and hand back real values once they exist, at the cost of blocking the GUI
// behind a progress bar while they arrive.
waitForTimeStepToArrive( result, stepIndex );
// Either the wait above delivered the values, or the transfer timed out / is still pending: read
// whatever is in the slot now (the real values, or still the placeholder), so a stalled transfer draws
// blank instead of hanging the GUI forever.
if ( auto* slot = resultValueSlot( result, stepIndex ); slot && !slot->empty() )
{
*values = *slot;
return true;
}
return fillWithUndefinedValues( values );
}
//--------------------------------------------------------------------------------------------------
/// See header. Writes into the case's result storage the same way an arrived async time step would, so a
/// later normal read finds it already there.
//--------------------------------------------------------------------------------------------------
bool RifReaderSumoGridProperty::prefetchDynamicResult( const QString& propertyName, const std::vector<size_t>& stepIndices )
{
if ( !m_caseData || stepIndices.empty() ) return false;
auto it = m_dynamicTimestamps.find( propertyName );
if ( it == m_dynamicTimestamps.end() ) return false;
const std::vector<QString>& timestamps = it->second;
auto* cellResults = m_caseData->results( RiaDefines::PorosityModelType::MATRIX_MODEL );
if ( !cellResults ) return false;
const RigEclipseResultAddress resultAddress( RiaDefines::ResultCatType::DYNAMIC_NATIVE, propertyName );
if ( !cellResults->hasResultEntry( resultAddress ) ) return false;
auto* timeStepValues = cellResults->modifiableCellScalarResultTimesteps( resultAddress );
if ( !timeStepValues ) return false;
// Size to the full time series, matching normal on-demand loading, so other time steps stay in bounds.
if ( timeStepValues->size() < timestamps.size() ) timeStepValues->resize( timestamps.size() );
// Only ask for what is not already there. requestTimeStepsAsync() itself skips whatever is already
// pending (e.g. requested by a live view of the same realization), so this is safe to call even when
// some of these steps are already on their way.
std::vector<size_t> stepsToRequest;
for ( size_t step : stepIndices )
{
if ( step >= timestamps.size() || timestamps[step].isEmpty() ) continue; // No data at this step.
if ( !( *timeStepValues )[step].empty() ) continue; // Already there.
stepsToRequest.push_back( step );
}
if ( !stepsToRequest.empty() ) requestTimeStepsAsync( propertyName, timestamps, stepsToRequest );
// Block until every requested step is resolved. All of them are already in flight together (or were
// already pending/loaded before this call), so waiting on them one at a time here does not serialize the
// transfers themselves - only this call's return.
bool allResolved = true;
for ( size_t step : stepIndices )
{
if ( step >= timestamps.size() || timestamps[step].isEmpty() ) continue;
waitForTimeStepToArrive( propertyName, step );
if ( ( *timeStepValues )[step].empty() ) allResolved = false;
}
// None of the requested steps has any Sumo data at all for this property (e.g. this realization's own
// reporting dates do not include any of the statistics' selected global dates for it), so nothing above
// was requested or written. Leaving every slot genuinely empty would leave
// RigCaseCellResultsData::isDataPresent() seeing this property as never having been looked at, so the
// next read of it (e.g. RigEclipseContourMapProjection::generateResults()'s own
// ensureKnownResultLoaded() call right after this) falls back to eagerly loading this property's entire
// time series instead of accepting "no data at the steps that were asked for" - exactly the unbounded
// fetch this function exists to avoid. Record that this property was genuinely looked at for this case by
// placing one real (if blank) placeholder, the same one an arrived-but-empty transfer would leave behind.
if ( stepsToRequest.empty() &&
std::none_of( timeStepValues->begin(), timeStepValues->end(), []( const std::vector<double>& v ) { return !v.empty(); } ) )
{
for ( size_t step : stepIndices )
{
if ( step < timeStepValues->size() )
{
fillWithUndefinedValues( &( *timeStepValues )[step] );
break;
}
}
}
return allResolved;
}
//--------------------------------------------------------------------------------------------------
/// Names what is on its way, so the user is told the cells are blank because data is being fetched and not
/// because there is none. A single time step is named outright; several are counted.
//--------------------------------------------------------------------------------------------------
QString RifReaderSumoGridProperty::pendingDataDescription() const
{
if ( m_pending.empty() ) return {};
std::set<QString> propertyNames;
for ( const auto& [propertyName, stepIndex] : m_pending )
{
propertyNames.insert( propertyName );
}
if ( m_pending.size() == 1 )
{
const auto& [propertyName, stepIndex] = *m_pending.begin();
QString isoDateOrInterval;
if ( auto it = m_dynamicTimestamps.find( propertyName ); it != m_dynamicTimestamps.end() && stepIndex < it->second.size() )
{
isoDateOrInterval = it->second[stepIndex];
}
if ( !isoDateOrInterval.isEmpty() ) return QString( "%1 (%2)" ).arg( propertyName, isoDateOrInterval );
return propertyName;
}
if ( propertyNames.size() == 1 )
{
return QString( "%1 time steps of %2" ).arg( m_pending.size() ).arg( *propertyNames.begin() );
}
return QString( "%1 time steps" ).arg( m_pending.size() );
}
//--------------------------------------------------------------------------------------------------
/// The transfer was issued by the case before this reader existed, so the time step was never marked pending
/// here. Claim it now, so the arrival handler treats it like any other.
//--------------------------------------------------------------------------------------------------
void RifReaderSumoGridProperty::acceptFetchedTimeStep( const QString& propertyName,
size_t stepIndex,
const QString& isoDateOrInterval,
const QByteArray& contents )
{
// Whichever transfer arrives first wins. Claim the step if it is not already claimed, so these values
// are written either way; a duplicate arriving later finds nothing pending and drops itself. Only bump
// the batch total when this call is the one actually claiming it, so a duplicate arrival does not count
// the same step twice.
if ( m_pending.insert( PendingKey{ propertyName, stepIndex } ).second ) m_batchProgressTotal++;
onTimeStepArrived( propertyName, stepIndex, isoDateOrInterval, contents );
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RifReaderSumoGridProperty::markTimeStepPending( const QString& propertyName, size_t stepIndex )
{
if ( !m_pending.insert( PendingKey{ propertyName, stepIndex } ).second ) return;
m_batchProgressTotal++;
updateBatchProgress();
notifyPendingChanged();
}
//--------------------------------------------------------------------------------------------------
/// Start a transfer for the time steps not already on their way, marking them pending before the request is
/// issued so a redraw arriving in between does not start the same transfer twice.
//--------------------------------------------------------------------------------------------------
void RifReaderSumoGridProperty::requestTimeStepsAsync( const QString& propertyName,
const std::vector<QString>& timestamps,
const std::vector<size_t>& steps )
{
if ( !m_connector ) return;
// No cap on how many are requested together: QNetworkAccessManager already limits how many of these
// actually run at once per host and queues the rest, so this reader only needs to avoid asking for the
// same step twice, not throttle the batch itself.
std::vector<QString> isoDatesOrIntervals;
std::vector<size_t> requestedSteps;
for ( size_t step : steps )
{
if ( step >= timestamps.size() || timestamps[step].isEmpty() ) continue;
if ( !m_pending.insert( PendingKey{ propertyName, step } ).second ) continue; // Already on its way.
requestedSteps.push_back( step );
isoDatesOrIntervals.push_back( timestamps[step] );
// Placeholder for every requested step, not just the displayed one: a failed prefetch step would
// otherwise leave its slot empty and no longer pending, indistinguishable from never requested, and
// get re-fetched on the next nearby redraw.
if ( auto* slot = resultValueSlot( propertyName, step ); slot && slot->empty() )
{
fillWithUndefinedValues( slot );
}
}
if ( isoDatesOrIntervals.empty() ) return;
// One dialog for the whole batch instead of one per waited-for step: grow the running total by however
// many genuinely new steps this call just added, then let updateBatchProgress open or resize the dialog.
m_batchProgressTotal += requestedSteps.size();
updateBatchProgress();
// Maps a delivered timestamp back to the time step it was requested for. Should a property report the
// same timestamp twice, the first one is the step that was asked for.
std::map<QString, size_t> stepByTimestamp;
for ( size_t i = 0; i < requestedSteps.size(); i++ )
{
stepByTimestamp.try_emplace( isoDatesOrIntervals[i], requestedSteps[i] );
}
notifyPendingChanged();
std::weak_ptr<bool> isAlive = m_lifetimeToken;
m_connector->grid().propertyDataBatchAsync(
SumoCaseId( m_caseId ),
m_ensembleName,
m_gridName,
m_realization,
propertyName,
isoDatesOrIntervals,
[this, isAlive, propertyName, stepByTimestamp]( const QString& isoDateOrInterval, const QByteArray& contents )
{
// The reader may be gone: a realization can be closed while its
// transfers are still running.
if ( isAlive.expired() ) return;
auto it = stepByTimestamp.find( isoDateOrInterval );
if ( it == stepByTimestamp.end() ) return;
onTimeStepArrived( propertyName, it->second, isoDateOrInterval, contents );
},
m_lifetimeToken.get() );
}
//--------------------------------------------------------------------------------------------------
/// Blocks the calling thread until the given time step is no longer pending or the transfer times out
/// (RiaSumoDefines::gridPropertyTransferTimeoutMillis()), pumping a local QEventLoop so the transfer itself
/// (which runs through Qt's own event loop on this same thread) can still make progress while this call
/// waits for it. A no-op if the step is not pending, e.g. it already arrived or failed before this call.
//--------------------------------------------------------------------------------------------------
void RifReaderSumoGridProperty::waitForTimeStepToArrive( const QString& propertyName, size_t stepIndex )
{
// Refresh here as well: a transfer started before the case was open (see markTimeStepPending) was
// registered while the view had no viewer yet, so this is the first point at which the owner can
// actually show it.
notifyPendingChanged();
if ( m_pending.count( PendingKey{ propertyName, stepIndex } ) == 0 ) return;
QEventLoop loop;
std::weak_ptr<bool> isAlive = m_lifetimeToken;
QTimer pollTimer;
QObject::connect( &pollTimer,
&QTimer::timeout,
[&]()
{
if ( isAlive.expired() || m_pending.count( PendingKey{ propertyName, stepIndex } ) == 0 ) loop.quit();
} );
pollTimer.start( 20 );
QTimer giveUpTimer;
giveUpTimer.setSingleShot( true );
QObject::connect( &giveUpTimer, &QTimer::timeout, &loop, &QEventLoop::quit );
giveUpTimer.start( RiaSumoDefines::gridPropertyTransferTimeoutMillis() );
loop.exec( QEventLoop::ProcessEventsFlag::ExcludeUserInputEvents );
}
//--------------------------------------------------------------------------------------------------
/// One time step has arrived, on the GUI thread. Write it into the case results over the placeholder, then
/// tell the owner via m_onTimeStepArrived, successfully or not, so it can redraw.
//--------------------------------------------------------------------------------------------------
void RifReaderSumoGridProperty::onTimeStepArrived( const QString& propertyName,
size_t stepIndex,
const QString& isoDateOrInterval,
const QByteArray& contents )
{
// No longer pending means it was abandoned while in flight, so these values are not wanted.
if ( m_pending.erase( PendingKey{ propertyName, stepIndex } ) == 0 ) return;
// Counts toward the single shared batch dialog whether this step succeeded or failed, and closes it once
// nothing is left pending.
m_batchProgressCompleted++;
updateBatchProgress();
if ( contents.isEmpty() )
{
RiaLogging::warning( QString( "Failed to load '%1' (time '%2') for realization %3 from Sumo." )
.arg( propertyName, isoDateOrInterval )
.arg( m_realization )
.toStdString() );
// The placeholder is left in place. A non-empty slot is not read again, so a failed step is not
// retried on every redraw, which would turn one failure into a flood of requests. The cells stay
// blank until the case is reloaded.
notifyPendingChanged();
notifyTimeStepArrived( propertyName, stepIndex, false );
return;
}
// Overwrite the placeholder: a non-empty slot means RigCaseCellResultsData does not ask for this time
// step again.
auto* slot = resultValueSlot( propertyName, stepIndex );
if ( !slot ) return;
if ( !decodeInto( contents, propertyName, slot ) ) return;
logTransfer( propertyName, isoDateOrInterval, contents.size(), true );
// The placeholder pass produced a degenerate min/max, and the legend range is built from it. Throw the
// cached statistics away so they are computed again from the values that just arrived.
if ( auto* cellResults = m_caseData ? m_caseData->results( RiaDefines::PorosityModelType::MATRIX_MODEL ) : nullptr )
{
cellResults->recalculateStatistics( RigEclipseResultAddress( RiaDefines::ResultCatType::DYNAMIC_NATIVE, propertyName ) );
}
notifyPendingChanged();
notifyTimeStepArrived( propertyName, stepIndex, true );
}
//--------------------------------------------------------------------------------------------------
/// Builds the "Loading X from Sumo..." message from pendingDataDescription() and reports it via
/// m_onPendingChanged. No GUI code here: the owner decides where and how to show it.
//--------------------------------------------------------------------------------------------------
void RifReaderSumoGridProperty::notifyPendingChanged() const
{
// Regression tests are left alone, matching how RiuMainWindow treats its own status messages.
if ( !m_onPendingChanged || RiaRegressionTestRunner::instance()->isRunningRegressionTests() ) return;
const QString pending = pendingDataDescription();
const QString message = pending.isEmpty() ? QString() : QString( "Loading %1 from Sumo..." ).arg( pending );
m_onPendingChanged( message );
}
//--------------------------------------------------------------------------------------------------
/// Reports how many of a batch's time steps are done via m_onBatchProgressChanged, instead of owning any
/// progress bar itself: called after requestTimeStepsAsync/markTimeStepPending/acceptFetchedTimeStep grow
/// the batch total, and after onTimeStepArrived counts one more step done. Reports total 0 once m_pending is
/// empty, so the owner knows the whole batch is over and can close whatever it is showing for it.
//--------------------------------------------------------------------------------------------------
void RifReaderSumoGridProperty::updateBatchProgress()
{
if ( m_pending.empty() )
{
// Whole batch done: tell the owner nothing is left, and reset the counters so the next batch starts
// counting from zero instead of continuing this one's total.
m_batchProgressTotal = 0;
m_batchProgressCompleted = 0;
if ( m_onBatchProgressChanged ) m_onBatchProgressChanged( 0, 0, QString() );
return;
}
if ( m_onBatchProgressChanged )
{
m_onBatchProgressChanged( m_batchProgressCompleted, m_batchProgressTotal, pendingDataDescription() );
}
}
//--------------------------------------------------------------------------------------------------
/// HUGE_VAL is the undefined-cell value the rest of the code already uses - the same value
/// RifRoffFileTools::propertyValuesFromStream writes for inactive cells - so a pending time step renders as
/// blank cells. A correctly sized vector is returned rather than an empty one, which the result accessors
/// would read out of range.
//--------------------------------------------------------------------------------------------------
bool RifReaderSumoGridProperty::fillWithUndefinedValues( std::vector<double>* values ) const
{
if ( !values || !m_caseData || !m_caseData->mainGrid() ) return false;
values->assign( m_caseData->mainGrid()->cellCount(), HUGE_VAL );
return true;
}
//--------------------------------------------------------------------------------------------------
/// Simply forwards to m_onTimeStepArrived. No redraw logic here: the owner decides what showing this data
/// means and how to guard against re-entrancy from a redraw that reads cell results and arrives back here.
//--------------------------------------------------------------------------------------------------
void RifReaderSumoGridProperty::notifyTimeStepArrived( const QString& propertyName, size_t stepIndex, bool ok ) const
{
if ( m_onTimeStepArrived ) m_onTimeStepArrived( propertyName, stepIndex, ok );
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
std::vector<size_t>
RifReaderSumoGridProperty::timeStepsToFetch( const QString& propertyName, const std::vector<QString>& timestamps, size_t stepIndex )
{
std::vector<size_t> steps{ stepIndex };
if ( !m_connector ) return steps;
// Request every other time step of this property not already loaded or on its way, alongside the one
// asked for, in the same batch transfer. QNetworkAccessManager caps how many of these actually run at
// once per host and queues the rest, so there is no need for this reader to also throttle how much is
// asked for - it only needs to avoid asking for the same step twice.
for ( size_t step = 0; step < timestamps.size(); step++ )
{
if ( step == stepIndex ) continue;
if ( timestamps[step].isEmpty() ) continue; // no data at this time step for this property
if ( m_pending.count( PendingKey{ propertyName, step } ) > 0 ) continue; // already on its way
auto* slot = resultValueSlot( propertyName, step );
if ( slot && !slot->empty() ) continue; // already loaded
steps.push_back( step );
}
return steps;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
std::vector<double>* RifReaderSumoGridProperty::resultValueSlot( const QString& propertyName, size_t stepIndex )
{
if ( !m_caseData ) return nullptr;
auto* cellResults = m_caseData->results( RiaDefines::PorosityModelType::MATRIX_MODEL );
if ( !cellResults ) return nullptr;
const RigEclipseResultAddress resultAddress( RiaDefines::ResultCatType::DYNAMIC_NATIVE, propertyName );
if ( !cellResults->hasResultEntry( resultAddress ) ) return nullptr;
auto* timeStepValues = cellResults->modifiableCellScalarResultTimesteps( resultAddress );
if ( !timeStepValues || stepIndex >= timeStepValues->size() ) return nullptr;
return &( ( *timeStepValues )[stepIndex] );
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
bool RifReaderSumoGridProperty::fetchAndDecode( const QString& propertyName, const QString& isoDateOrInterval, std::vector<double>* values )
{
if ( !m_connector || !m_caseData || !values ) return false;
QByteArray contents =
m_connector->grid().propertyData( SumoCaseId( m_caseId ), m_ensembleName, m_gridName, m_realization, propertyName, isoDateOrInterval );
logTransfer( propertyName, isoDateOrInterval, contents.size(), false );
return decodeInto( contents, propertyName, values );
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RifReaderSumoGridProperty::logTransfer( const QString& propertyName, const QString& isoDateOrInterval, size_t byteCount, bool fromBatch ) const
{
RiaLogging::debug( std::format( "Sumo grid property '{}' realization {} (time '{}') [{}]: {} bytes",
propertyName.toStdString(),
m_realization,
isoDateOrInterval.toStdString(),
fromBatch ? "batch" : "single",
byteCount ) );
}
//--------------------------------------------------------------------------------------------------
/// Decode a downloaded roff property blob into cell values. The same post processing as the on demand
/// path in RigCaseCellResultsData is applied, so a prefetched time step is indistinguishable from one
/// read when it was asked for.
//--------------------------------------------------------------------------------------------------
bool RifReaderSumoGridProperty::decodeInto( const QByteArray& contents, const QString& propertyName, std::vector<double>* values )
{
if ( contents.isEmpty() || !m_caseData || !values ) return false;
std::string buffer = contents.toStdString();
std::istringstream stream( buffer, std::ios::binary );
if ( !RifRoffFileTools::propertyValuesFromStream( stream, m_caseData, propertyName, values ) ) return false;
return true;
}
@@ -0,0 +1,189 @@
/////////////////////////////////////////////////////////////////////////////////
//
// Copyright (C) 2024 Equinor ASA
//
// ResInsight is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// ResInsight is distributed in the hope that it will be useful, but WITHOUT ANY
// WARRANTY; without even the implied warranty of MERCHANTABILITY or
// FITNESS FOR A PARTICULAR PURPOSE.
//
// See the GNU General Public License at <http://www.gnu.org/licenses/gpl.html>
// for more details.
//
/////////////////////////////////////////////////////////////////////////////////
#pragma once
#include "RifReaderInterface.h"
#include <QByteArray>
#include <QPointer>
#include <QString>
#include <functional>
#include <map>
#include <memory>
#include <set>
#include <utility>
#include <vector>
class RiaSumoConnector;
//==================================================================================================
//
// Lazily fetches grid cell properties for a single Sumo grid realization. ResInsight's result
// machinery calls staticResult()/dynamicResult() the first time a property is displayed; this reader
// then downloads the corresponding roff blob from Sumo and decodes the cell values.
//
//==================================================================================================
class RifReaderSumoGridProperty : public RifReaderInterface
{
public:
// Called whenever what is pending changes, with a user-facing message describing it (empty when nothing
// is pending). No GUI code lives here: the owner decides where and how to show it.
using PendingChangedCallback = std::function<void( const QString& )>;
// Called once a requested time step is done, successfully or not, so the owner can redraw whatever
// shows this data.
using TimeStepArrivedCallback = std::function<void( const QString&, size_t, bool )>;
// Called whenever how many of a batch's time steps are done changes: completed, total, and a description
// of what is currently being transferred (matches pendingDataDescription()). total is 0 once nothing is
// pending, meaning the batch is over. No GUI code lives here: the owner decides whether and how to show a
// progress bar from these numbers.
using BatchProgressChangedCallback = std::function<void( size_t /*completed*/, size_t /*total*/, const QString& /*description*/ )>;
RifReaderSumoGridProperty( RiaSumoConnector* connector,
const QString& caseId,
const QString& ensembleName,
const QString& gridName,
int realization );
// Aborts any transfers still in flight for this reader, see RiaSumoConnector::cancelGroup, so they stop
// consuming connections and logging results for a reader that is no longer there.
~RifReaderSumoGridProperty() override;
void setPendingChangedCallback( PendingChangedCallback callback );
void setTimeStepArrivedCallback( TimeStepArrivedCallback callback );
void setBatchProgressChangedCallback( BatchProgressChangedCallback callback );
void setStaticProperties( const std::vector<QString>& propertyNames );
void setDynamicProperties( const std::map<QString, std::vector<QString>>& propertyNameToTimestamps );
bool open( const QString& fileName, RigEclipseCaseData* eclipseCase ) override;
// Take one time step fetched before this reader existed, see RimRoffCaseSumo::startPropertyFetch.
void acceptFetchedTimeStep( const QString& propertyName, size_t stepIndex, const QString& isoDateOrInterval, const QByteArray& contents );
// Record a transfer started before this reader existed as already on its way. Without this the reader
// reports nothing in flight, so the user is told nothing while waiting and a second, duplicate transfer
// is issued the first time the property is read.
void markTimeStepPending( const QString& propertyName, size_t stepIndex );
// What is being transferred right now, for display to the user. Empty when nothing is.
QString pendingDataDescription() const;
bool staticResult( const QString& result, RiaDefines::PorosityModelType matrixOrFracture, std::vector<double>* values ) override;
bool dynamicResult( const QString& result, RiaDefines::PorosityModelType matrixOrFracture, size_t stepIndex, std::vector<double>* values ) override;
// Fetches and stores exactly the given time steps of a dynamic property - no other time step of it is
// pulled in. Meant for a caller with no redraw loop of its own that needs a specific set of time steps,
// e.g. RimStatisticsContourMap: having them already present keeps
// RigCaseCellResultsData::ensureKnownResultLoaded() from falling back to loading the whole time series.
// The given steps are requested together as one async batch (so the transfers run in parallel, the same
// as dynamicResult()'s own look-ahead batch), and this call blocks until every one of them has either
// arrived or timed out before returning - so by the time it returns, nothing requested by this call is
// left in flight. Returns false if any requested step never got a value (missing, failed, or timed out).
bool prefetchDynamicResult( const QString& propertyName, const std::vector<size_t>& stepIndices );
private:
bool fetchAndDecode( const QString& propertyName, const QString& isoDateOrInterval, std::vector<double>* values );
bool decodeInto( const QByteArray& contents, const QString& propertyName, std::vector<double>* values );
// One line per transferred time step, naming the realization and whether it came from a batch or from
// a single request. This reader is per realization, so without the realization a run of single
// requests from several readers is indistinguishable from one reader fetching the same steps twice.
void logTransfer( const QString& propertyName, const QString& isoDateOrInterval, size_t byteCount, bool fromBatch ) const;
// The time steps to download together with stepIndex: stepIndex itself, plus every other unloaded step
// of this property not already pending or loaded, so the whole property can be requested in one batch.
// Which of these actually run at once is left to QNetworkAccessManager - this reader does not throttle
// the batch itself.
std::vector<size_t> timeStepsToFetch( const QString& propertyName, const std::vector<QString>& timestamps, size_t stepIndex );
// Where the case keeps the values of one time step of a dynamic property, or null when the property is
// not registered as a result. Writing here is what makes a prefetched time step available without
// holding on to the downloaded bytes.
std::vector<double>* resultValueSlot( const QString& propertyName, size_t stepIndex );
// One requested time step of one dynamic property.
using PendingKey = std::pair<QString, size_t>;
// Request the given time steps without waiting, marking them pending. Already pending steps are skipped,
// so a redraw mid-transfer does not issue a second request. Not capped: every given step not already
// pending is requested in the same batch, relying on QNetworkAccessManager to bound how many actually
// run at once and queue the rest.
void requestTimeStepsAsync( const QString& propertyName, const std::vector<QString>& timestamps, const std::vector<size_t>& steps );
// Blocks the calling thread until the given time step is no longer pending or the transfer times out
// (RiaSumoDefines::gridPropertyTransferTimeoutMillis()). Lets dynamicResult() hand back the real values
// instead of a placeholder, so callers never see partially-loaded data - the same blocking contract a
// disk-based reader already has. No GUI code here: whatever progress bar the owner shows via
// m_onBatchProgressChanged runs independently, driven by requestTimeStepsAsync/onTimeStepArrived. A no-op
// if the step is not currently pending.
void waitForTimeStepToArrive( const QString& propertyName, size_t stepIndex );
// Called on the connector thread, which is the GUI thread, once per requested time step.
void onTimeStepArrived( const QString& propertyName, size_t stepIndex, const QString& isoDateOrInterval, const QByteArray& contents );
// Fill values with the undefined-cell value, sized to the grid, and report success, so a time step still
// on its way draws blank instead of logging a load failure on every redraw.
bool fillWithUndefinedValues( std::vector<double>* values ) const;
// Reports the time step is done via m_onTimeStepArrived, successfully or not, so the owner can redraw.
void notifyTimeStepArrived( const QString& propertyName, size_t stepIndex, bool ok ) const;
// Reports what is pending via m_onPendingChanged, built from pendingDataDescription().
void notifyPendingChanged() const;
// Reports how many of a batch's time steps are done via m_onBatchProgressChanged: opens/grows the count
// the first time something becomes pending (called from requestTimeStepsAsync/markTimeStepPending/
// acceptFetchedTimeStep) and ticks it as each step arrives (called from onTimeStepArrived), reporting
// total 0 once m_pending is empty again so the owner knows the batch is over. No GUI code here: the
// owner decides whether and how to show a progress bar from these numbers.
void updateBatchProgress();
private:
QPointer<RiaSumoConnector> m_connector;
QString m_caseId;
QString m_ensembleName;
QString m_gridName;
int m_realization;
RigEclipseCaseData* m_caseData; // set in open(); used for cell count and active cell masking
std::vector<QString> m_staticProperties;
std::map<QString, std::vector<QString>> m_dynamicTimestamps; // property name -> sorted iso timestamps
// Requested and not yet arrived. Claimed before the request is issued, so it also answers "already on
// its way?".
std::set<PendingKey> m_pending;
// Dropped when this reader dies. Callbacks hold a weak_ptr and return early once it expires, so a reply
// outliving the reader cannot write into freed memory.
std::shared_ptr<bool> m_lifetimeToken;
PendingChangedCallback m_onPendingChanged;
TimeStepArrivedCallback m_onTimeStepArrived;
BatchProgressChangedCallback m_onBatchProgressChanged;
// Bookkeeping only, no GUI: how many time steps have been added to the current batch and how many of
// those have arrived (successfully or not), reported to the owner via m_onBatchProgressChanged. Both
// reset to 0 once m_pending empties again.
size_t m_batchProgressTotal = 0;
size_t m_batchProgressCompleted = 0;
};
@@ -108,6 +108,14 @@ bool RifRoffFileTools::openGridFile( const QString& fileName, RigEclipseCaseData
return false;
}
return openGridFile( stream, eclipseCase, errorMessages );
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
bool RifRoffFileTools::openGridFile( std::istream& stream, RigEclipseCaseData* eclipseCase, QString* errorMessages )
{
auto getInt = []( auto values, const std::string& name )
{
auto v = std::find_if( values.begin(), values.end(), [&name]( const auto& arg ) { return arg.first == name; } );
@@ -274,6 +282,7 @@ bool RifRoffFileTools::openGridFile( const QString& fileName, RigEclipseCaseData
catch ( std::runtime_error& err )
{
RiaLogging::error( std::format( "Roff file import failed: {}", err.what() ) );
if ( errorMessages ) *errorMessages = QString::fromStdString( err.what() );
return false;
}
@@ -492,17 +501,26 @@ std::pair<bool, std::map<QString, QString>> RifRoffFileTools::createInputPropert
{
RiaLogging::info( std::format( "Reading properties from roff file: {}", fileName ) );
std::string filename = fileName.toStdString();
std::map<QString, QString> keywordMapping;
std::ifstream stream( filename, std::ios::binary );
std::ifstream stream( fileName.toStdString(), std::ios::binary );
if ( !stream.good() )
{
RiaLogging::error( "Unable to open roff file" );
return std::make_pair( false, keywordMapping );
return std::make_pair( false, std::map<QString, QString>{} );
}
return createInputProperties( stream, eclipseCaseData, fileName );
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
std::pair<bool, std::map<QString, QString>> RifRoffFileTools::createInputProperties( std::istream& stream,
RigEclipseCaseData* eclipseCaseData,
const QString& sourceName,
RiaDefines::ResultCatType resultCategory )
{
std::map<QString, QString> keywordMapping;
auto codeNamesAndValuesForKeyword = []( const std::string& keyword, roff::Reader& reader ) -> std::map<int, QString>
{
const std::string codeNamesKeyword = keyword + roff::Parser::postFixCodeNames();
@@ -545,7 +563,7 @@ std::pair<bool, std::map<QString, QString>> RifRoffFileTools::createInputPropert
{
if ( !appendZoneIndexPropertyFromSubgrids( eclipseCaseData, reader, keywordMapping ) )
{
RiaLogging::warning( std::format( "Unable to import ROFF subgrids zonation from {}", fileName ) );
RiaLogging::warning( std::format( "Unable to import ROFF subgrids zonation from {}", sourceName ) );
}
}
else if ( eclipseCaseData->mainGrid()->cellCount() == keywordLength )
@@ -558,9 +576,9 @@ std::pair<bool, std::map<QString, QString>> RifRoffFileTools::createInputPropert
newResultName = "ACTNUM";
}
if ( !appendNewInputPropertyResult( eclipseCaseData, newResultName, keyword, kind, reader ) )
if ( !appendNewInputPropertyResult( eclipseCaseData, newResultName, keyword, kind, reader, resultCategory ) )
{
RiaLogging::error( std::format( "Unable to import result '{}' from {}", keyword, fileName ) );
RiaLogging::error( std::format( "Unable to import result '{}' from {}", keyword, sourceName ) );
return std::make_pair( false, keywordMapping );
}
@@ -604,6 +622,15 @@ std::pair<bool, std::map<QString, QString>> RifRoffFileTools::createInputPropert
const auto codeNames = codeNamesAndValuesForKeyword( keyword, reader );
RicFaciesPropertiesImportTools::createColorLegendMatchDefaultRockColors( codeNames );
}
else
{
// Skipped: typically grid metadata, but a property array whose length does not match the grid
// cell count is also skipped here. Log it so a size mismatch does not fail silently.
RiaLogging::debug( std::format( "Skipping roff array '{}' (length {}), grid cell count is {}.",
keyword,
keywordLength,
eclipseCaseData->mainGrid()->cellCount() ) );
}
}
}
catch ( std::runtime_error& err )
@@ -612,6 +639,13 @@ std::pair<bool, std::map<QString, QString>> RifRoffFileTools::createInputPropert
return std::make_pair( false, keywordMapping );
}
if ( keywordMapping.empty() )
{
RiaLogging::warning( std::format( "No grid properties matching the grid cell count ({}) were imported from {}.",
eclipseCaseData->mainGrid()->cellCount(),
sourceName.toStdString() ) );
}
return std::make_pair( true, keywordMapping );
}
@@ -680,11 +714,90 @@ std::vector<double>
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
bool RifRoffFileTools::appendNewInputPropertyResult( RigEclipseCaseData* caseData,
const QString& resultName,
const std::string& keyword,
roff::Token::Kind kind,
roff::Reader& reader )
bool RifRoffFileTools::propertyValuesFromStream( std::istream& stream,
RigEclipseCaseData* eclipseCaseData,
const QString& propertyName,
std::vector<double>* values )
{
if ( !eclipseCaseData || !eclipseCaseData->mainGrid() || !values ) return false;
auto mainGrid = eclipseCaseData->mainGrid();
const int nx = static_cast<int>( mainGrid->cellCountI() );
const int ny = static_cast<int>( mainGrid->cellCountJ() );
const int nz = static_cast<int>( mainGrid->cellCountK() );
const size_t cellCount = mainGrid->cellCount();
try
{
roff::Reader reader( stream );
reader.parse();
// Collect the arrays whose length matches the grid cell count. A Sumo property blob may contain more
// than one such array, so pick the one whose keyword matches the requested property name rather than
// blindly taking the first match (which could decode a different property).
std::vector<std::pair<std::string, roff::Token::Kind>> candidates;
for ( const auto& [keyword, kind] : reader.getNamedArrayTypes() )
{
if ( reader.getArrayLength( keyword ) == cellCount ) candidates.push_back( { keyword, kind } );
}
if ( candidates.empty() ) return false;
std::string candidateNames;
for ( const auto& [keyword, kind] : candidates )
{
if ( !candidateNames.empty() ) candidateNames += ", ";
candidateNames += keyword;
}
auto selected = candidates.front();
for ( const auto& candidate : candidates )
{
if ( QString::fromStdString( candidate.first ).compare( propertyName, Qt::CaseInsensitive ) == 0 )
{
selected = candidate;
break;
}
}
RiaLogging::debug( std::format( "Roff property '{}': arrays matching cell count [{}], using '{}'.",
propertyName.toStdString(),
candidateNames,
selected.first ) );
std::vector<double> roffValues = readAndConvertToDouble( nx, ny, nz, selected.first, selected.second, reader );
if ( roffValues.size() != cellCount ) return false;
// Set better invalid value for inactive cells: roff file has -999.
auto activeCellInfo = eclipseCaseData->activeCellInfo( RiaDefines::PorosityModelType::MATRIX_MODEL );
for ( size_t i = 0; i < cellCount; i++ )
{
if ( !activeCellInfo->isActive( ReservoirCellIndex( mainGrid->reservoirCellIndex( i ) ) ) )
{
roffValues[i] = HUGE_VAL;
}
}
*values = std::move( roffValues );
return true;
}
catch ( std::runtime_error& err )
{
RiaLogging::error( std::format( "Roff property parsing failed: {}", err.what() ) );
return false;
}
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
bool RifRoffFileTools::appendNewInputPropertyResult( RigEclipseCaseData* caseData,
const QString& resultName,
const std::string& keyword,
roff::Token::Kind kind,
roff::Reader& reader,
RiaDefines::ResultCatType resultCategory )
{
CAF_ASSERT( caseData );
@@ -707,7 +820,7 @@ bool RifRoffFileTools::appendNewInputPropertyResult( RigEclipseCaseData* caseDat
}
}
RigEclipseResultAddress resAddr( RiaDefines::ResultCatType::INPUT_PROPERTY, RifRoffFileTools::mapFromType( kind ), resultName );
RigEclipseResultAddress resAddr( resultCategory, RifRoffFileTools::mapFromType( kind ), resultName );
caseData->results( RiaDefines::PorosityModelType::MATRIX_MODEL )->createResultEntry( resAddr, false );
auto newPropertyData = caseData->results( RiaDefines::PorosityModelType::MATRIX_MODEL )->modifiableCellScalarResultTimesteps( resAddr );
@@ -25,6 +25,7 @@
#include <QString>
#include <iosfwd>
#include <map>
#include <vector>
@@ -52,10 +53,31 @@ public:
static bool openGridFile( const QString& fileName, RigEclipseCaseData* eclipseCase, QString* errorMessages );
// Parse a roff grid directly from an already opened binary stream. Used when the roff data does not
// originate from a file on disk (e.g. a blob downloaded from Sumo).
static bool openGridFile( std::istream& stream, RigEclipseCaseData* eclipseCase, QString* errorMessages );
static std::pair<bool, std::map<QString, QString>> createInputProperties( const QString& fileName, RigEclipseCaseData* eclipseCase );
// Read roff property data from an already opened binary stream (e.g. a blob downloaded from Sumo). The
// sourceName is only used for log messages. resultCategory selects the result category the properties are
// imported into (e.g. STATIC_NATIVE for static and DYNAMIC_NATIVE for time dependent properties).
static std::pair<bool, std::map<QString, QString>>
createInputProperties( std::istream& stream,
RigEclipseCaseData* eclipseCase,
const QString& sourceName,
RiaDefines::ResultCatType resultCategory = RiaDefines::ResultCatType::INPUT_PROPERTY );
static bool hasGridData( const QString& filename );
// Read a single grid property from an in-memory roff blob and return its values (with inactive cells
// masked), without registering it. Among the arrays matching the grid cell count, the one whose keyword
// matches propertyName (case-insensitive) is preferred; otherwise the first matching array is used.
static bool propertyValuesFromStream( std::istream& stream,
RigEclipseCaseData* eclipseCase,
const QString& propertyName,
std::vector<double>* values );
static size_t computeActiveCellMatrixIndex( std::vector<int>& activeCells );
static std::vector<double> computeZoneValuesFromSubgrids( const std::vector<int>& nLayers, size_t nx, size_t ny, size_t nz );
@@ -81,11 +103,12 @@ private:
static std::vector<double>
readAndConvertToDouble( int nx, int ny, int nz, const std::string& keyword, roff::Token::Kind kind, roff::Reader& reader );
static bool appendNewInputPropertyResult( RigEclipseCaseData* caseData,
const QString& resultName,
const std::string& keyword,
roff::Token::Kind token,
roff::Reader& reader );
static bool appendNewInputPropertyResult( RigEclipseCaseData* caseData,
const QString& resultName,
const std::string& keyword,
roff::Token::Kind token,
roff::Reader& reader,
RiaDefines::ResultCatType resultCategory );
static bool
appendZoneIndexPropertyFromSubgrids( RigEclipseCaseData* caseData, roff::Reader& reader, std::map<QString, QString>& keywordMapping );
@@ -11,6 +11,7 @@ set(SOURCE_GROUP_SOURCE_FILES
${CMAKE_CURRENT_LIST_DIR}/RimEclipseInputPropertyCollection.cpp
${CMAKE_CURRENT_LIST_DIR}/RimEclipseInputCase.cpp
${CMAKE_CURRENT_LIST_DIR}/RimEclipseResultCase.cpp
${CMAKE_CURRENT_LIST_DIR}/RimRoffCaseSumo.cpp
${CMAKE_CURRENT_LIST_DIR}/RimEclipseView.cpp
${CMAKE_CURRENT_LIST_DIR}/RimEclipseResultDefinition.cpp
${CMAKE_CURRENT_LIST_DIR}/RimEclipseCellColors.cpp
@@ -129,6 +130,7 @@ set(SOURCE_GROUP_SOURCE_FILES
${CMAKE_CURRENT_LIST_DIR}/RimCornerPointCase.cpp
${CMAKE_CURRENT_LIST_DIR}/RimReservoirGridEnsemble.cpp
${CMAKE_CURRENT_LIST_DIR}/RimReservoirGridEnsembleBase.cpp
${CMAKE_CURRENT_LIST_DIR}/RimReservoirGridEnsembleSumo.cpp
)
list(APPEND CODE_SOURCE_FILES ${SOURCE_GROUP_SOURCE_FILES})
@@ -1,5 +1,6 @@
set(SOURCE_GROUP_SOURCE_FILES
${CMAKE_CURRENT_LIST_DIR}/RimCloudDataSourceCollection.cpp
${CMAKE_CURRENT_LIST_DIR}/RimSumoDataSource.cpp
)
list(APPEND CODE_SOURCE_FILES ${SOURCE_GROUP_SOURCE_FILES})
@@ -18,16 +18,18 @@
#include "RimCloudDataSourceCollection.h"
#include "RiaLogging.h"
#include "Cloud/RiaCloudApiService.h"
#include "RiaApplication.h"
#include "Summary/RiaSummaryPlotTools.h"
#include "Summary/RiaSummaryTools.h"
#include "Cloud/RimSummaryEnsembleSumo.h"
#include "RimOilField.h"
#include "RimProject.h"
#include "RimSummaryCaseMainCollection.h"
#include "Sumo/RimSummaryEnsembleSumo.h"
#include "Sumo/RimSumoDataSource.h"
#include "RimSumoDataSource.h"
#include "RiuPlotMainWindowTools.h"
@@ -93,6 +95,73 @@ std::vector<RimSumoDataSource*> RimCloudDataSourceCollection::sumoDataSources()
return m_sumoDataSources.childrenByType();
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RimCloudDataSourceCollection::initAfterRead()
{
refreshDataSourcesFromSumo();
}
//--------------------------------------------------------------------------------------------------
/// The realization ids, vector names and grid names of a data source are not written to the project file,
/// only the identity of the ensemble and the realization filter are. A data source read back from a project
/// therefore has none of them, which leaves the realization filter without the set it filters. Ask Sumo for
/// them again here, so the data source is as complete as one just added.
//--------------------------------------------------------------------------------------------------
void RimCloudDataSourceCollection::refreshDataSourcesFromSumo()
{
if ( !m_sumoConnector ) return;
for ( auto dataSource : sumoDataSources() )
{
if ( !dataSource ) continue;
const auto caseId = dataSource->caseId();
const auto ensembleName = dataSource->ensembleName();
if ( caseId.get().isEmpty() || ensembleName.isEmpty() ) continue;
// Only what is missing is asked for, and an empty answer is not stored: a failed request must not
// replace values that are already there. Note that Sumo answering with nothing is indistinguishable
// from the request failing, so an ensemble genuinely without realizations is asked about again on the
// next load. That costs a request and is preferable to recording "no realizations" after a failure.
if ( !dataSource->hasFetchedRealizations() )
{
if ( const auto realizationIds = m_sumoConnector->explore().realizationIds( caseId, ensembleName ); !realizationIds.empty() )
{
dataSource->setAvailableRealizationIds( realizationIds );
}
else
{
// Without the realizations the realization filter has nothing to filter, and editing it
// cannot be told from deselecting everything. Say so once, here, rather than leaving the
// data source looking empty for no stated reason.
RiaLogging::warning( QString( "Unable to load the realizations of Sumo ensemble '%1'. Editing the "
"realization filter will have no effect until they are available." )
.arg( ensembleName )
.toStdString() );
continue;
}
}
if ( dataSource->vectorNames().empty() )
{
if ( const auto vectorNames = m_sumoConnector->summary().vectorNames( caseId, ensembleName ); !vectorNames.empty() )
{
dataSource->setVectorNames( vectorNames );
}
}
if ( dataSource->gridNames().empty() )
{
if ( const auto gridNames = m_sumoConnector->grid().gridNames( caseId, ensembleName ); !gridNames.empty() )
{
dataSource->setGridNames( gridNames );
}
}
}
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
@@ -464,6 +533,7 @@ std::vector<RimSumoDataSource*> RimCloudDataSourceCollection::addDataSources()
}
const auto availableRealizationIds = m_sumoConnector->explore().realizationIds( sumoCaseId, ensembleName );
const auto gridNames = m_sumoConnector->grid().gridNames( sumoCaseId, ensembleName );
const auto vectorNames = m_sumoConnector->summary().vectorNames( sumoCaseId, ensembleName );
auto dataSource = new RimSumoDataSource();
@@ -473,6 +543,13 @@ std::vector<RimSumoDataSource*> RimCloudDataSourceCollection::addDataSources()
dataSource->setEnsembleName( ensembleName );
dataSource->setAvailableRealizationIds( availableRealizationIds );
dataSource->setVectorNames( vectorNames );
dataSource->setGridNames( gridNames );
if ( !gridNames.empty() )
{
dataSource->setSelectedGridName( gridNames.front() );
}
dataSource->updateName();
objectToSelect = dataSource;
@@ -47,6 +47,12 @@ public:
static void createEnsemblesFromSelectedDataSources( const std::vector<RimSumoDataSource*>& dataSources );
private:
void initAfterRead() override;
// Fetch the realization ids, vector names and grid names of the data sources that do not have them. They
// are not written to the project file, so a data source restored from one starts out without them.
void refreshDataSourcesFromSumo();
void fieldChangedByUi( const caf::PdmFieldHandle* changedField, const QVariant& oldValue, const QVariant& newValue ) override;
QList<caf::PdmOptionItemInfo> calculateValueOptions( const caf::PdmFieldHandle* fieldNeedingOptions ) override;
@@ -1,6 +1,6 @@
/////////////////////////////////////////////////////////////////////////////////
//
// Copyright (C) 2024 Equinor ASA
// Copyright (C) 2026 Equinor ASA
//
// ResInsight is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
@@ -18,9 +18,16 @@
#include "RimSumoDataSource.h"
#include "RiaLogging.h"
#include "RiaStdStringTools.h"
#include "RimSummaryEnsembleSumo.h"
#include "Cloud/RimSummaryEnsembleSumo.h"
#include "Rim3dOverlayInfoConfig.h"
#include "RimEclipseView.h"
#include "RimEclipseViewCollection.h"
#include "RimProject.h"
#include "RimReservoirGridEnsembleSumo.h"
#include "RimRoffCaseSumo.h"
#include "cafCmdFeatureMenuBuilder.h"
#include "cafPdmPointer.h"
@@ -29,7 +36,10 @@
#include <QCoreApplication>
CAF_PDM_SOURCE_INIT( RimSumoDataSource, "RimSumoDataSource", "RimSummarySumoDataSource" );
#include <algorithm>
#include <set>
CAF_PDM_SOURCE_INIT( RimSumoDataSource, "RimSumoDataSource", "RimSummarySumoDataSource", "RimSumoGridDataSource" );
//--------------------------------------------------------------------------------------------------
///
@@ -46,6 +56,7 @@ RimSumoDataSource::RimSumoDataSource()
CAF_PDM_InitFieldNoDefault( &m_availableRealizationIds, "RealizationIds", "Available Realization Ids" );
m_availableRealizationIds.uiCapability()->setUiHidden( true );
m_availableRealizationIds.xmlCapability()->disableIO();
CAF_PDM_InitField( &m_realizationFilter,
"RealizationFilter",
@@ -61,6 +72,21 @@ RimSumoDataSource::RimSumoDataSource()
CAF_PDM_InitFieldNoDefault( &m_vectorNames, "VectorNames", "Vector Names" );
m_vectorNames.uiCapability()->setUiHidden( true );
m_vectorNames.xmlCapability()->disableIO();
CAF_PDM_InitFieldNoDefault( &m_gridNames, "GridNames", "Grid Names" );
m_gridNames.uiCapability()->setUiHidden( true );
m_gridNames.xmlCapability()->disableIO();
CAF_PDM_InitFieldNoDefault( &m_selectedGridName, "GridName", "Grid Name" );
CAF_PDM_InitField( &m_doComputeMobileVolumeWeightedMean,
"DoComputeMobileVolumeWeightedMean",
false,
"Compute Mobile Volume Weighted Mean",
"",
"Show the mobile volume weighted mean in the 3D info box. This downloads the PORV and SWCR grid "
"properties for every realization." );
setDeletable( true );
}
@@ -137,12 +163,21 @@ std::vector<QString> RimSumoDataSource::availableRealizationIds() const
return m_availableRealizationIds();
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
bool RimSumoDataSource::hasFetchedRealizations() const
{
return m_hasFetchedRealizations;
}
//--------------------------------------------------------------------------------------------------
/// The available realizations are the source of truth. An empty filter selects all of them.
//--------------------------------------------------------------------------------------------------
void RimSumoDataSource::setAvailableRealizationIds( const std::vector<QString>& realizationIds )
{
m_availableRealizationIds = realizationIds;
m_hasFetchedRealizations = true;
// Show the full range instead of an empty field. Only an empty filter is replaced, keeping a user
// or project defined filter. Selection is unchanged, as empty and full range both select all.
@@ -198,6 +233,46 @@ void RimSumoDataSource::setVectorNames( const std::vector<QString>& vectorNames
m_vectorNames = vectorNames;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
std::vector<QString> RimSumoDataSource::gridNames() const
{
return m_gridNames();
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RimSumoDataSource::setGridNames( const std::vector<QString>& gridNames )
{
m_gridNames = gridNames;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
QString RimSumoDataSource::selectedGridName() const
{
return m_selectedGridName();
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RimSumoDataSource::setSelectedGridName( const QString& gridName )
{
m_selectedGridName = gridName;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
bool RimSumoDataSource::doComputeMobileVolumeWeightedMean() const
{
return m_doComputeMobileVolumeWeightedMean();
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
@@ -220,6 +295,7 @@ void RimSumoDataSource::updateName()
void RimSumoDataSource::appendMenuItems( caf::CmdFeatureMenuBuilder& menuBuilder ) const
{
menuBuilder.addCmdFeature( "RicCreateSumoEnsembleFeature" );
menuBuilder.addCmdFeature( "RicCreateSumoReservoirGridEnsembleFeature" );
}
//--------------------------------------------------------------------------------------------------
@@ -270,6 +346,28 @@ void RimSumoDataSource::defineUiOrdering( QString uiConfigName, caf::PdmUiOrderi
auto ensembleGroup = uiOrdering.addNewGroup( "Ensemble Selection" );
ensembleGroup->add( &m_realizationFilter );
ensembleGroup->add( &m_realizationFilterInfo );
auto gridGroup = uiOrdering.addNewGroup( "Grid Selection" );
gridGroup->add( &m_selectedGridName );
gridGroup->add( &m_doComputeMobileVolumeWeightedMean );
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
QList<caf::PdmOptionItemInfo> RimSumoDataSource::calculateValueOptions( const caf::PdmFieldHandle* fieldNeedingOptions )
{
QList<caf::PdmOptionItemInfo> options;
if ( fieldNeedingOptions == &m_selectedGridName )
{
for ( const auto& gridName : m_gridNames() )
{
options.push_back( caf::PdmOptionItemInfo( gridName, gridName ) );
}
}
return options;
}
//--------------------------------------------------------------------------------------------------
@@ -285,6 +383,10 @@ void RimSumoDataSource::fieldChangedByUi( const caf::PdmFieldHandle* changedFiel
{
onRealizationFilterChanged();
}
else if ( changedField == &m_doComputeMobileVolumeWeightedMean )
{
updateGridCaseEnsembles();
}
}
//--------------------------------------------------------------------------------------------------
@@ -299,6 +401,145 @@ void RimSumoDataSource::onRealizationFilterChanged()
{
ensemble->onRealizationSelectionChanged();
}
// Update any grid case ensembles created from this data source.
updateGridCaseEnsembles();
}
//--------------------------------------------------------------------------------------------------
/// Synchronize the realization grid cases (RimRoffCaseSumo) of every grid ensemble created from this
/// data source with the current realization filter: remove cases for deselected realizations and add
/// cases for newly selected ones. The grid name of each ensemble is preserved.
//--------------------------------------------------------------------------------------------------
void RimSumoDataSource::updateGridCaseEnsembles()
{
// Synchronizing against realizations that have not been fetched would read "not known yet" as "nothing
// selected" and remove the realization cases of every grid ensemble. That is what a project load followed
// by an edit of the realization filter would otherwise do.
const bool realizationsAreKnown = hasFetchedRealizations();
std::set<int> selectedRealizations;
for ( const auto& realizationId : selectedRealizationIds() )
{
bool ok = false;
int value = realizationId.toInt( &ok );
if ( ok ) selectedRealizations.insert( value );
}
// The ensembles hold the back link to this data source, so they are found even when the realization
// filter has left them without any cases.
for ( auto ensemble : objectsWithReferringPtrFieldsOfType<RimReservoirGridEnsembleSumo>() )
{
std::vector<RimRoffCaseSumo*> gridCases;
for ( auto eclipseCase : ensemble->sourceCases() )
{
if ( auto gridCase = dynamic_cast<RimRoffCaseSumo*>( eclipseCase ) ) gridCases.push_back( gridCase );
}
// All cases in an ensemble share the same grid, and the ensemble knows which one, so the name is
// available also when the filter has left it without any cases.
const QString gridName = ensemble->sumoGridName();
if ( gridName.isEmpty() ) continue;
if ( ensemble->doComputeMobileVolumeWeightedMean() != m_doComputeMobileVolumeWeightedMean() )
{
ensemble->setDoComputeMobileVolumeWeightedMean( m_doComputeMobileVolumeWeightedMean() );
// The 3D info box reads the setting when it is updated, so refresh it to apply the change immediately.
if ( auto viewColl = ensemble->viewCollection() )
{
for ( auto view : viewColl->views() )
{
if ( view->overlayInfoConfig() ) view->overlayInfoConfig()->update3DInfo();
}
}
}
// Everything above applies whether or not the realizations are known. Adding and removing cases below
// does not, so leave the ensemble as it is until the realizations have been fetched.
if ( !realizationsAreKnown ) continue;
// A realization the grid does not exist for would only produce a case failing to load. An empty list
// means the realizations of the grid are not known, then do not filter on them.
std::set<int> realizationsToSelect = selectedRealizations;
if ( const auto gridRealizations = ensemble->gridRealizations(); !gridRealizations.empty() )
{
std::erase_if( realizationsToSelect,
[&gridRealizations]( int realization )
{ return std::ranges::find( gridRealizations, realization ) == gridRealizations.end(); } );
}
// The selection can be changed after the ensemble was created, so report the realizations left out
// here as well, not only when the ensemble is created. The ensemble reports the current set in its
// property panel and marks itself in the tree, see RimReservoirGridEnsembleSumo.
if ( const auto missing = ensemble->realizationsWithoutGridData(); !missing.empty() )
{
RiaLogging::warning( QString( "Grid '%1' has no data for %2 of the selected realizations, no cases created "
"for them: %3" )
.arg( gridName )
.arg( missing.size() )
.arg( QString::fromStdString( RiaStdStringTools::formatRangeSelection( missing ) ) )
.toStdString() );
}
std::set<int> currentRealizations;
for ( auto gridCase : gridCases )
{
currentRealizations.insert( gridCase->realization() );
}
// Add cases for newly selected realizations first, so a view displaying a removed realization
// can be repointed to one of these instead of being deleted.
for ( int realization : realizationsToSelect )
{
if ( currentRealizations.find( realization ) != currentRealizations.end() ) continue;
if ( auto* gridCase = RimRoffCaseSumo::createFromDataSource( this, gridName, realization ) )
{
ensemble->addCase( gridCase );
if ( auto project = RimProject::current() ) project->assignCaseIdToCase( gridCase );
}
}
// Remove cases for realizations no longer selected. Remember any views that displayed them, so
// they can be repointed to a surviving case below (a deleted case auto-nulls the view's case,
// which would otherwise make the view disappear, e.g. when changing 30-40 to 31-40).
std::vector<RimEclipseView*> orphanedViews;
for ( auto gridCase : gridCases )
{
if ( realizationsToSelect.find( gridCase->realization() ) != realizationsToSelect.end() ) continue;
if ( auto viewColl = ensemble->viewCollection() )
{
for ( auto view : viewColl->views() )
{
if ( view->eclipseCase() == gridCase ) orphanedViews.push_back( view );
}
}
ensemble->removeCase( gridCase );
delete gridCase;
}
// Repoint orphaned views to a surviving case, or delete them if no realizations remain.
const auto remainingCases = ensemble->sourceCases();
for ( auto view : orphanedViews )
{
if ( !remainingCases.empty() )
{
view->setEclipseCase( remainingCases.front() );
view->loadDataAndUpdate();
}
else if ( auto viewColl = ensemble->viewCollection() )
{
viewColl->removeView( view );
delete view;
}
}
ensemble->updateConnectedEditors();
}
}
//--------------------------------------------------------------------------------------------------
@@ -25,12 +25,12 @@
//==================================================================================================
//
// Common data source describing a single Sumo ensemble. Holds the information required by summary
// ensembles (asset, vector names). The available ensemble realizations - fetched from the
// realizations endpoint - are the source of truth, and the user selects a subset of them ("ensemble
// selection"). Consumers listen to the selected realization id subset. All values are populated by
// RimCloudDataSourceCollection, which owns the RiaSumoConnector; this object does not talk to the
// connector directly.
// Common data source describing a single Sumo ensemble. Holds the information required both for
// summary ensembles (vector names) and for grid case ensembles (asset, grid names). The available
// ensemble realizations - fetched from the realizations endpoint - are the source of truth, and the
// user selects a subset of them ("ensemble selection"). Both summary and grid consumers listen to
// the selected realization id subset. All values are populated by RimCloudDataSourceCollection,
// which owns the RiaSumoConnector; this object does not talk to the connector directly.
//
//==================================================================================================
@@ -57,7 +57,15 @@ public:
std::vector<QString> availableRealizationIds() const;
void setAvailableRealizationIds( const std::vector<QString>& realizationIds );
// The subset of realizations matching the realization filter. Summary ensemble creation listens to this.
// Whether the realizations have been fetched from Sumo. They are not written to the project file, so a
// data source read back from one starts out without them, see
// RimCloudDataSourceCollection::refreshDataSourcesFromSumo. Ask this rather than testing
// availableRealizationIds() for emptiness: an empty list cannot tell "not fetched" from "fetched, and the
// ensemble has none", and reading the first as the second removes the realization cases of every ensemble
// created from this data source.
bool hasFetchedRealizations() const;
// The subset of realizations matching the realization filter. Both summary and grid creation listen to this.
std::vector<QString> selectedRealizationIds() const;
// Available summary vectors for the ensemble. Not shown in the UI, but used to populate the
@@ -65,15 +73,25 @@ public:
std::vector<QString> vectorNames() const;
void setVectorNames( const std::vector<QString>& vectorNames );
std::vector<QString> gridNames() const;
void setGridNames( const std::vector<QString>& gridNames );
QString selectedGridName() const;
void setSelectedGridName( const QString& gridName );
bool doComputeMobileVolumeWeightedMean() const;
void updateName();
private:
void appendMenuItems( caf::CmdFeatureMenuBuilder& menuBuilder ) const override;
void defineEditorAttribute( const caf::PdmFieldHandle* field, QString uiConfigName, caf::PdmUiEditorAttribute* attribute ) override;
void defineUiOrdering( QString uiConfigName, caf::PdmUiOrdering& uiOrdering ) override;
QList<caf::PdmOptionItemInfo> calculateValueOptions( const caf::PdmFieldHandle* fieldNeedingOptions ) override;
void fieldChangedByUi( const caf::PdmFieldHandle* changedField, const QVariant& oldValue, const QVariant& newValue ) override;
void onRealizationFilterChanged();
void updateGridCaseEnsembles();
QString realizationFilterInfoText() const;
QString availableRealizationsRangeText() const;
@@ -86,8 +104,16 @@ private:
caf::PdmField<QString> m_customName;
caf::PdmField<std::vector<QString>> m_availableRealizationIds;
caf::PdmField<QString> m_realizationFilter;
caf::PdmProxyValueField<QString> m_realizationFilterInfo;
// Runtime only, deliberately not a PdmField: it describes whether this session has fetched the
// realizations, which is never true for a data source just read from a project file.
bool m_hasFetchedRealizations = false;
caf::PdmField<QString> m_realizationFilter;
caf::PdmProxyValueField<QString> m_realizationFilterInfo;
caf::PdmField<std::vector<QString>> m_vectorNames;
caf::PdmField<std::vector<QString>> m_gridNames;
caf::PdmField<QString> m_selectedGridName;
caf::PdmField<bool> m_doComputeMobileVolumeWeightedMean;
};
@@ -185,6 +185,8 @@ RimStatisticsContourMap::RimStatisticsContourMap()
m_resultDefinition->findField( "MResultType" )->uiCapability()->setUiName( "Result" );
m_resultDefinition->setResultType( RiaDefines::ResultCatType::DYNAMIC_NATIVE );
m_resultDefinition->setResultVariable( "SOIL" );
// Only a settings picker here; actual results are read per realization in computeStatisticsForMaps().
m_resultDefinition->setEagerResultLoadingEnabled( false );
CAF_PDM_InitFieldNoDefault( &m_primaryCase,
"PrimaryEclipseCase",
@@ -755,6 +757,34 @@ void RimStatisticsContourMap::computeStatisticsForMaps( const std::vector<RimSta
auto activeCellInfo = eclipseCaseData->activeCellInfo( RiaDefines::PorosityModelType::MATRIX_MODEL );
auto resultData = eclipseCaseData->results( RiaDefines::PorosityModelType::MATRIX_MODEL );
// Prefetch every dynamic result time step this case needs before reading any of it for real.
// This case has no view of its own, so it is the only chance a cloud-backed reader gets to
// fetch exactly the time steps needed before the case is closed again - see
// RimEclipseCase::prefetchDynamicResult. A no-op for a case that reads from disk. Grouped by
// result name so every time step a property needs is requested together as one parallel
// batch (rather than one blocking round trip per time step); prefetchDynamicResult() itself
// blocks until that whole batch has arrived before returning, so nothing it requested is
// left in flight when this case is closed again below - no separate wait is needed here.
std::map<QString, std::vector<size_t>> stepsToPrefetchByResult;
for ( auto& ctx : contexts )
{
if ( !ctx.active || !ctx.map->m_resultDefinition()->hasDynamicResult() ) continue;
auto& steps = stepsToPrefetchByResult[ctx.map->m_resultDefinition()->eclipseResultAddress().resultName()];
for ( auto [localTs, globalTs] : ctx.map->mapLocalToGlobalTimeSteps( eCase->timeStepDates() ) )
{
steps.push_back( static_cast<size_t>( localTs ) );
}
}
for ( auto& [resultName, steps] : stepsToPrefetchByResult )
{
std::sort( steps.begin(), steps.end() );
steps.erase( std::unique( steps.begin(), steps.end() ), steps.end() );
eCase->prefetchDynamicResult( resultName, steps );
}
for ( auto& ctx : contexts )
{
if ( !ctx.active ) continue;
@@ -101,6 +101,8 @@ public:
std::vector<QString> selectedFormations() const;
std::vector<std::vector<cvf::Vec3d>> selectedPolygons() const;
void switchToSelectedSourceCase();
protected:
void defineUiOrdering( QString uiConfigName, caf::PdmUiOrdering& uiOrdering ) override;
void fieldChangedByUi( const caf::PdmFieldHandle* changedField, const QVariant& oldValue, const QVariant& newValue ) override;
@@ -111,8 +113,6 @@ protected:
QList<caf::PdmOptionItemInfo> calculateValueOptions( const caf::PdmFieldHandle* fieldNeedingOptions ) override;
void onChildDeleted( caf::PdmChildArrayFieldHandle* childArray, std::vector<caf::PdmObjectHandle*>& referringObjects ) override;
void switchToSelectedSourceCase();
private:
using TimestepResultsMap = std::map<size_t, std::vector<std::vector<double>>>;
@@ -56,6 +56,7 @@
#include "RimGeoMechView.h"
#include "RimGridView.h"
#include "RimReservoirCellResultsStorage.h"
#include "RimReservoirGridEnsembleBase.h"
#include "RimSeismicData.h"
#include "RimSeismicView.h"
#include "RimSimWellInViewCollection.h"
@@ -162,6 +163,10 @@ RigHistogramData Rim3dOverlayInfoConfig::histogramData()
auto geoMechContourMap = dynamic_cast<RimGeoMechContourMapView*>( geoMechView );
auto seismicView = dynamic_cast<RimSeismicView*>( m_viewDef.p() );
// The mobile volume weighted mean requires MOBPORV (PORV, SWCR and MULTPV). Skip the calculation when the value
// is not displayed, as computing it can be expensive for cases backed by remote data.
m_histogramCalculator->setDoComputeMobileVolumeWeightedMean( m_showVolumeWeightedMean() );
if ( eclipseContourMap )
return m_histogramCalculator->histogramData( eclipseContourMap );
else if ( geoMechContourMap )
@@ -754,6 +759,18 @@ void Rim3dOverlayInfoConfig::update3DInfo()
{
m_showVolumeWeightedMean = false;
}
if ( auto eclipseCase = reservoirView->eclipseCase() )
{
// Ensembles can opt out of the mobile volume weighted mean, as the data it is derived from can be
// expensive to fetch. Used by Sumo grid ensembles, see RimSumoDataSource.
auto ensemble = eclipseCase->firstAncestorOrThisOfType<RimReservoirGridEnsembleBase>();
if ( ensemble && !ensemble->doComputeMobileVolumeWeightedMean() )
{
m_showVolumeWeightedMean = false;
}
}
updateEclipse3DInfo( reservoirView );
}
@@ -884,6 +901,17 @@ void Rim3dOverlayInfoConfig::updateEclipse3DInfo( RimEclipseView* eclipseView )
infoText += resultInfoText( histData );
}
// Not behind m_showCaseInfo or m_showResultInfo: blank cells with no explanation are exactly what this
// is here to prevent, so it is shown whenever a transfer is running.
if ( auto eclipseCase = eclipseView->eclipseCase() )
{
const QString loadingText = eclipseCase->dataLoadingText();
if ( !loadingText.isEmpty() )
{
infoText += QString( "<p><b>%1...</b></p>" ).arg( loadingText );
}
}
if ( !infoText.isEmpty() )
{
eclipseView->viewer()->setInfoText( infoText );
@@ -80,8 +80,9 @@ public:
RimEclipseCase();
~RimEclipseCase() override;
std::vector<RimEclipseView*> reservoirViews() const;
RimEclipseViewCollection* viewCollection() const;
std::vector<RimEclipseView*> reservoirViews() const;
std::vector<RimEclipseContourMapView*> contourMapViews() const;
RimEclipseViewCollection* viewCollection() const;
std::vector<QString> filesContainingFaults() const;
void setFilesContainingFaults( const std::vector<QString>& val );
@@ -177,15 +178,27 @@ protected:
RimEclipseViewCollection* globalViewCollection() const;
void addViewsFromViewCollection( std::vector<RimEclipseView*>& views, const RimEclipseViewCollection* viewColl ) const;
const RigFormationNames* effectiveFormationNames() const;
public:
// What this case is transferring right now, for the 3D view overlay. Empty for a case that reads from
// disk and has nothing in flight; overridden by the cases backed by remote data.
virtual QString dataLoadingText() const { return {}; }
// Fetches and stores exactly the given time steps of a dynamic property, in parallel, without pulling in
// any other time step. A no-op returning false for a case that reads from disk; overridden by a
// cloud-backed case (e.g. RimRoffCaseSumo) so a caller with no view/redraw loop of its own - such as
// ensemble statistics - can warm up precisely the time steps it needs before reading results, without
// downloading anything else.
virtual bool prefetchDynamicResult( const QString& /*resultName*/, const std::vector<size_t>& /*stepIndices*/ ) { return false; }
protected:
private:
void createTimeStepFormatString();
std::vector<Rim3dView*> allSpecialViews() const override;
std::vector<RimEclipseContourMapView*> contourMapViews() const;
void createTimeStepFormatString();
std::vector<Rim3dView*> allSpecialViews() const override;
void buildResultChildNodes();
const RigFormationNames* effectiveFormationNames() const;
protected:
caf::PdmField<bool> m_flipXAxis;
caf::PdmField<bool> m_flipYAxis;
@@ -51,10 +51,18 @@ RimEclipseCaseEnsemble::RimEclipseCaseEnsemble()
CAF_PDM_InitFieldNoDefault( &m_viewCollection, "ViewCollection", "Views" );
m_viewCollection = new RimEclipseViewCollection;
// Limit the case dropdown of the ensemble views to the cases of this ensemble.
m_viewCollection->setEclipseCaseProvider( [this]() { return this->cases(); } );
CAF_PDM_InitFieldNoDefault( &m_wellTargetMappings, "WellTargetMappings", "Well Target Mappings" );
CAF_PDM_InitFieldNoDefault( &m_statisticsContourMaps, "StatisticsContourMaps", "Statistics Contour maps" );
// Set by the data source creating the ensemble. Defaults to true to keep the behavior of ensembles created
// before this field was introduced.
CAF_PDM_InitField( &m_doComputeMobileVolumeWeightedMean, "DoComputeMobileVolumeWeightedMean", true, "Compute Mobile Volume Weighted Mean" );
m_doComputeMobileVolumeWeightedMean.uiCapability()->setUiHidden( true );
setDeletable( true );
}
@@ -70,6 +78,18 @@ RimEclipseCaseEnsemble::~RimEclipseCaseEnsemble()
m_viewCollection = nullptr;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RimEclipseCaseEnsemble::initAfterRead()
{
// The provider is a lambda and not serialized, so re-apply it after loading a project.
if ( m_viewCollection )
{
m_viewCollection->setEclipseCaseProvider( [this]() { return this->cases(); } );
}
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
@@ -274,6 +294,22 @@ std::vector<RimStatisticsContourMap*> RimEclipseCaseEnsemble::statisticsContourM
return m_statisticsContourMaps.childrenByType();
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
bool RimEclipseCaseEnsemble::doComputeMobileVolumeWeightedMean() const
{
return m_doComputeMobileVolumeWeightedMean();
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RimEclipseCaseEnsemble::setDoComputeMobileVolumeWeightedMean( bool enable )
{
m_doComputeMobileVolumeWeightedMean = enable;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
@@ -53,8 +53,8 @@ public:
GridModeType gridMode() const override;
QString ensembleName() const override;
void addCase( RimEclipseCase* reservoir );
void removeCase( RimEclipseCase* reservoir );
void addCase( RimEclipseCase* reservoir ) override;
void removeCase( RimEclipseCase* reservoir ) override;
bool contains( RimEclipseCase* reservoir ) const;
RimEclipseCase* findByFileName( const QString& gridFileName ) const;
@@ -67,7 +67,7 @@ public:
std::vector<RimEclipseView*> allViews() const;
RimEclipseViewCollection* viewCollection() const;
RimEclipseViewCollection* viewCollection() const override;
void addWellTargetMapping( RimWellTargetMapping* wellTargetMapping );
std::vector<RimWellTargetMapping*> wellTargetMappings() const;
@@ -75,7 +75,11 @@ public:
void addStatisticsContourMap( RimStatisticsContourMap* statisticsContourMap ) override;
std::vector<RimStatisticsContourMap*> statisticsContourMaps() const override;
bool doComputeMobileVolumeWeightedMean() const override;
void setDoComputeMobileVolumeWeightedMean( bool enable ) override;
protected:
void initAfterRead() override;
void appendMenuItems( caf::CmdFeatureMenuBuilder& menuBuilder ) const override;
void defineUiOrdering( QString uiConfigName, caf::PdmUiOrdering& uiOrdering ) override;
@@ -86,4 +90,5 @@ private:
caf::PdmChildArrayField<RimWellTargetMapping*> m_wellTargetMappings;
caf::PdmChildArrayField<RimStatisticsContourMap*> m_statisticsContourMaps;
caf::PdmPtrField<RimEclipseCase*> m_selectedCase;
caf::PdmField<bool> m_doComputeMobileVolumeWeightedMean;
};
@@ -95,6 +95,7 @@ RimEclipseResultDefinition::RimEclipseResultDefinition( caf::PdmUiItemInfo::Labe
: m_isDeltaResultEnabled( false )
, m_labelPosition( labelPosition )
, m_ternaryEnabled( true )
, m_eagerResultLoadingEnabled( true )
{
CAF_PDM_InitScriptableObjectWithNameAndComment( "Result Definition", "", "", "", "EclipseResult", "An eclipse result definition" );
@@ -1095,6 +1096,14 @@ int RimEclipseResultDefinition::caseDiffIndex() const
return -1;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RimEclipseResultDefinition::setEagerResultLoadingEnabled( bool enabled )
{
m_eagerResultLoadingEnabled = enabled;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
@@ -1102,6 +1111,8 @@ void RimEclipseResultDefinition::loadResult()
{
if ( isFlowDiagOrInjectionFlooding() ) return; // Will load automatically on access
if ( !m_eagerResultLoadingEnabled ) return; // Settings-only picker with no result ever displayed: nothing to load yet.
if ( m_eclipseCase )
{
if ( !m_eclipseCase->ensureReservoirCaseIsOpen() )
@@ -103,6 +103,7 @@ public:
QString additionalResultText() const;
virtual void loadResult();
void setEagerResultLoadingEnabled( bool enabled );
RigEclipseResultAddress eclipseResultAddress() const;
void setFromEclipseResultAddress( const RigEclipseResultAddress& resultAddress );
bool hasStaticResult() const;
@@ -200,6 +201,10 @@ private:
caf::PdmField<bool> m_divideByCellFaceArea;
caf::PdmField<bool> m_showDualPorosityLabel;
// Off for definitions that only pick settings and never display a result, so a selection does not
// eagerly load every time step of a result (expensive for cloud-backed cases).
bool m_eagerResultLoadingEnabled;
private:
void assignFlowSolutionFromCase();
@@ -201,6 +201,12 @@ bool RimEclipseStatisticsCase::openEclipseGridFile()
RigMainGrid* mainGrid = ensembleBase->mainGrid();
if ( !mainGrid ) return false;
// Statistics are computed from every realization, and the realizations of an ensemble can be opened
// lazily. Compute the union first, so it is complete before it is used below: an empty union would
// give a degenerate active cell bounding box and make computeCachedData() classify the faults of the
// shared grid from an all inactive grid.
ensembleBase->computeUnionOfActiveCells();
cvf::ref<RigEclipseCaseData> eclipseCase = new RigEclipseCaseData( this );
eclipseCase->setMainGrid( mainGrid );
@@ -91,6 +91,7 @@
#include "RimRegularLegendConfig.h"
#include "RimReservoirCellResultsStorage.h"
#include "RimReservoirGridEnsemble.h"
#include "RimRoffCaseSumo.h"
#include "RimSeismicSection.h"
#include "RimSeismicSectionCollection.h"
#include "RimSimWellInView.h"
@@ -526,6 +527,18 @@ void RimEclipseView::fieldChangedByUi( const caf::PdmFieldHandle* changedField,
if ( changedField == &m_eclipseCase )
{
// The field is already updated to the new case at this point. Cancel any in-flight grid and property
// downloads for the realization just switched away from, so they stop competing with the newly
// selected realization's transfers for bandwidth and connections, which could otherwise make the new
// realization's grid download fail. Already loaded data is kept, so switching back later does not
// force a full reload. Skipped when another view still shows it.
auto* previousCase = dynamic_cast<RimEclipseCase*>( oldValue.value<caf::PdmPointer<PdmObjectHandle>>().rawPtr() );
auto* previousSumoCase = dynamic_cast<RimRoffCaseSumo*>( previousCase );
if ( previousSumoCase && previousSumoCase->reservoirViews().empty() && previousSumoCase->contourMapViews().empty() )
{
previousSumoCase->cancelPendingTransfers();
}
propagateEclipseCaseToChildObjects();
// Sync with RimReloadCaseTools::updateAll3dViews
@@ -75,6 +75,7 @@ void caf::AppEnum<RimHistogramCalculator::StatisticsCellRangeType>::setUp()
RimHistogramCalculator::RimHistogramCalculator()
: m_isVisCellStatUpToDate( false )
, m_numBins( RigStatisticsDataCache::defaultNumBins() )
, m_doComputeMobileVolumeWeightedMean( true )
{
}
@@ -141,6 +142,14 @@ void RimHistogramCalculator::applyCustomBinning( RigStatisticsDataCache* statist
}
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RimHistogramCalculator::setDoComputeMobileVolumeWeightedMean( bool enable )
{
m_doComputeMobileVolumeWeightedMean = enable;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
@@ -272,7 +281,8 @@ RigHistogramData RimHistogramCalculator::histogramData( RimEclipseView*
fldResults->p10p90ScalarValues( resAddr, timeStep, &histData.p10, &histData.p90 );
fldResults->meanScalarValue( resAddr, timeStep, &histData.mean );
fldResults->sumScalarValue( resAddr, timeStep, &histData.sum );
fldResults->mobileVolumeWeightedMean( resAddr, timeStep, &histData.weightedMean );
if ( m_doComputeMobileVolumeWeightedMean )
fldResults->mobileVolumeWeightedMean( resAddr, timeStep, &histData.weightedMean );
histData.histogram = fldResults->scalarValuesHistogram( resAddr, timeStep );
@@ -290,7 +300,8 @@ RigHistogramData RimHistogramCalculator::histogramData( RimEclipseView*
m_visibleCellStatistics->minMaxCellScalarValues( timeStep, histData.min, histData.max );
m_visibleCellStatistics->p10p90CellScalarValues( timeStep, histData.p10, histData.p90 );
m_visibleCellStatistics->sumCellScalarValues( timeStep, histData.sum );
m_visibleCellStatistics->mobileVolumeWeightedMean( timeStep, histData.weightedMean );
if ( m_doComputeMobileVolumeWeightedMean )
m_visibleCellStatistics->mobileVolumeWeightedMean( timeStep, histData.weightedMean );
histData.histogram = m_visibleCellStatistics->cellScalarValuesHistogram( timeStep );
@@ -310,7 +321,7 @@ RigHistogramData RimHistogramCalculator::histogramData( RimEclipseView*
fldResults->p10p90ScalarValues( resAddr, &histData.p10, &histData.p90 );
fldResults->meanScalarValue( resAddr, &histData.mean );
fldResults->sumScalarValue( resAddr, &histData.sum );
fldResults->mobileVolumeWeightedMean( resAddr, &histData.weightedMean );
if ( m_doComputeMobileVolumeWeightedMean ) fldResults->mobileVolumeWeightedMean( resAddr, &histData.weightedMean );
histData.histogram = fldResults->scalarValuesHistogram( resAddr );
@@ -327,7 +338,7 @@ RigHistogramData RimHistogramCalculator::histogramData( RimEclipseView*
m_visibleCellStatistics->minMaxCellScalarValues( histData.min, histData.max );
m_visibleCellStatistics->p10p90CellScalarValues( histData.p10, histData.p90 );
m_visibleCellStatistics->sumCellScalarValues( histData.sum );
m_visibleCellStatistics->mobileVolumeWeightedMean( histData.weightedMean );
if ( m_doComputeMobileVolumeWeightedMean ) m_visibleCellStatistics->mobileVolumeWeightedMean( histData.weightedMean );
histData.histogram = m_visibleCellStatistics->cellScalarValuesHistogram();
@@ -348,7 +359,7 @@ RigHistogramData RimHistogramCalculator::histogramData( RimEclipseView*
cellResults->p10p90CellScalarValues( eclResAddr, histData.p10, histData.p90 );
cellResults->meanCellScalarValues( eclResAddr, histData.mean );
cellResults->sumCellScalarValues( eclResAddr, histData.sum );
cellResults->mobileVolumeWeightedMean( eclResAddr, histData.weightedMean );
if ( m_doComputeMobileVolumeWeightedMean ) cellResults->mobileVolumeWeightedMean( eclResAddr, histData.weightedMean );
histData.histogram = cellResults->cellScalarValuesHistogram( eclResAddr );
statisticsCache = cellResults->statistics( eclResAddr );
@@ -360,7 +371,8 @@ RigHistogramData RimHistogramCalculator::histogramData( RimEclipseView*
cellResults->p10p90CellScalarValues( eclResAddr, timeStep, histData.p10, histData.p90 );
cellResults->meanCellScalarValues( eclResAddr, timeStep, histData.mean );
cellResults->sumCellScalarValues( eclResAddr, timeStep, histData.sum );
cellResults->mobileVolumeWeightedMean( eclResAddr, timeStep, histData.weightedMean );
if ( m_doComputeMobileVolumeWeightedMean )
cellResults->mobileVolumeWeightedMean( eclResAddr, timeStep, histData.weightedMean );
histData.histogram = cellResults->cellScalarValuesHistogram( eclResAddr, timeStep );
statisticsCache = cellResults->statistics( eclResAddr );
@@ -380,7 +392,7 @@ RigHistogramData RimHistogramCalculator::histogramData( RimEclipseView*
m_visibleCellStatistics->minMaxCellScalarValues( histData.min, histData.max );
m_visibleCellStatistics->p10p90CellScalarValues( histData.p10, histData.p90 );
m_visibleCellStatistics->sumCellScalarValues( histData.sum );
m_visibleCellStatistics->mobileVolumeWeightedMean( histData.weightedMean );
if ( m_doComputeMobileVolumeWeightedMean ) m_visibleCellStatistics->mobileVolumeWeightedMean( histData.weightedMean );
histData.histogram = m_visibleCellStatistics->cellScalarValuesHistogram();
@@ -393,7 +405,7 @@ RigHistogramData RimHistogramCalculator::histogramData( RimEclipseView*
m_visibleCellStatistics->minMaxCellScalarValues( timeStep, histData.min, histData.max );
m_visibleCellStatistics->p10p90CellScalarValues( timeStep, histData.p10, histData.p90 );
m_visibleCellStatistics->sumCellScalarValues( timeStep, histData.sum );
m_visibleCellStatistics->mobileVolumeWeightedMean( timeStep, histData.weightedMean );
if ( m_doComputeMobileVolumeWeightedMean ) m_visibleCellStatistics->mobileVolumeWeightedMean( timeStep, histData.weightedMean );
histData.histogram = m_visibleCellStatistics->cellScalarValuesHistogram( timeStep );
@@ -63,6 +63,10 @@ public:
RigHistogramCalculator::OutOfRangeHandling outOfRangeHandling,
std::optional<std::pair<double, double>> customBinRange );
// The mobile volume weighted mean is derived from MOBPROV, which in turn requires PORV, SWCR and MULTPV. For
// cases backed by remote data these are expensive to fetch, so the calculation can be turned off.
void setDoComputeMobileVolumeWeightedMean( bool enable );
RigHistogramData histogramData( RimEclipseContourMapView* contourMap );
RigHistogramData histogramData( RimGeoMechContourMapView* contourMap );
RigHistogramData histogramData( RimEclipseView* eclipseView, StatisticsCellRangeType cellRange, StatisticsTimeRangeType timeRange );
@@ -93,4 +97,5 @@ private:
RigHistogramCalculator::BinningMode m_binningMode = RigHistogramCalculator::BinningMode::LINEAR;
RigHistogramCalculator::OutOfRangeHandling m_outOfRangeHandling = RigHistogramCalculator::OutOfRangeHandling::EXCLUDE;
std::optional<std::pair<double, double>> m_customBinRange;
bool m_doComputeMobileVolumeWeightedMean;
};
@@ -53,8 +53,8 @@ public:
caf::PdmField<int> groupId;
caf::PdmChildField<RimCaseCollection*> caseCollection;
void addCase( RimEclipseCase* reservoir );
void removeCase( RimEclipseCase* reservoir );
void addCase( RimEclipseCase* reservoir ) override;
void removeCase( RimEclipseCase* reservoir ) override;
bool contains( RimEclipseCase* reservoir ) const;
@@ -1500,6 +1500,10 @@ void RimProject::defineUiTreeOrdering( caf::PdmUiTreeOrdering& uiTreeOrdering, Q
if ( oilField )
{
if ( oilField->analysisModels() ) uiTreeOrdering.add( oilField->analysisModels() );
// Also shown in the plot window, see "PlotWindow.Cloud". A data source serves both windows: summary
// ensembles are created from it there, grid ensembles here, where the grid is going to be used.
if ( oilField->cloudDataCollection() ) uiTreeOrdering.add( oilField->cloudDataCollection() );
}
uiTreeOrdering.add( &m_ensembleFileSetCollection );
}
@@ -78,7 +78,6 @@ void caf::AppEnum<RimReservoirGridEnsembleBase::GridModeType>::setUp()
///
//--------------------------------------------------------------------------------------------------
RimReservoirGridEnsemble::RimReservoirGridEnsemble()
: m_mainGrid( nullptr )
{
CAF_PDM_InitScriptableObjectWithNameAndComment( "Reservoir Grid Ensemble",
":/GridCaseGroup16x16.png",
@@ -131,6 +130,14 @@ RimReservoirGridEnsemble::RimReservoirGridEnsemble()
setDeletable( true );
}
//--------------------------------------------------------------------------------------------------
/// Defined here (rather than defaulted in the header) so that the cvf::ref<RigMainGrid> member is
/// destroyed where RigMainGrid is a complete type.
//--------------------------------------------------------------------------------------------------
RimReservoirGridEnsemble::~RimReservoirGridEnsemble()
{
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
@@ -187,14 +194,14 @@ void RimReservoirGridEnsemble::addCase( RimEclipseCase* reservoir )
{
CAF_ASSERT( reservoir );
if ( !m_mainGrid && reservoir->eclipseCaseData() )
if ( m_mainGrid.isNull() && reservoir->eclipseCaseData() )
{
m_mainGrid = reservoir->eclipseCaseData()->mainGrid();
}
else if ( hasSharedGrid() && reservoir->eclipseCaseData() )
{
// Share the main grid for identical grids
reservoir->eclipseCaseData()->setMainGrid( m_mainGrid );
reservoir->eclipseCaseData()->setMainGrid( m_mainGrid.p() );
}
m_caseCollection()->reservoirs().push_back( reservoir );
@@ -296,7 +303,7 @@ bool RimReservoirGridEnsemble::hasSharedGrid() const
//--------------------------------------------------------------------------------------------------
bool RimReservoirGridEnsemble::isGridDataLoaded() const
{
return m_mainGrid != nullptr;
return m_mainGrid.notNull();
}
//--------------------------------------------------------------------------------------------------
@@ -322,12 +329,51 @@ QString RimReservoirGridEnsemble::ensembleName() const
RigMainGrid* RimReservoirGridEnsemble::mainGrid()
{
// Trigger deferred loading if needed
if ( !m_mainGrid && !cases().empty() )
if ( m_mainGrid.isNull() && !cases().empty() )
{
const_cast<RimReservoirGridEnsemble*>( this )->loadGridDataFromFiles();
loadGridDataFromFiles();
}
return m_mainGrid;
return m_mainGrid.p();
}
//--------------------------------------------------------------------------------------------------
/// True when the two grids have the same IJK dimensions, which is what makes it safe to use one of them
/// in place of the other. The total cell count also guards against a grid with LGRs.
//--------------------------------------------------------------------------------------------------
bool RimReservoirGridEnsemble::hasMatchingDimensions( const RigMainGrid* lhs, const RigMainGrid* rhs )
{
if ( !lhs || !rhs ) return false;
return lhs->cellCountI() == rhs->cellCountI() && lhs->cellCountJ() == rhs->cellCountJ() && lhs->cellCountK() == rhs->cellCountK() &&
lhs->totalCellCount() == rhs->totalCellCount();
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
RigMainGrid* RimReservoirGridEnsemble::shareOrAdoptMainGrid( RigMainGrid* candidate )
{
if ( !candidate || !hasSharedGrid() ) return candidate;
// The first realization to be opened provides the grid the rest of the ensemble shares.
if ( m_mainGrid.isNull() )
{
m_mainGrid = candidate;
return candidate;
}
// The dimensions are checked when the ensemble is created, so this only triggers when the meta data
// and the grid itself disagree. Let the realization keep its own grid rather than failing to open it.
if ( !hasMatchingDimensions( m_mainGrid.p(), candidate ) )
{
RiaLogging::warning( std::format( "Grid ensemble '{}': grid dimensions differ from the shared grid. "
"Keeping a separate grid for this realization.",
name().toStdString() ) );
return candidate;
}
return m_mainGrid.p();
}
//--------------------------------------------------------------------------------------------------
@@ -351,7 +397,7 @@ void RimReservoirGridEnsemble::setupSharedGrid()
{
if ( allCases[i] && allCases[i]->eclipseCaseData() )
{
allCases[i]->eclipseCaseData()->setMainGrid( m_mainGrid );
allCases[i]->eclipseCaseData()->setMainGrid( m_mainGrid.p() );
}
}
}
@@ -371,6 +417,33 @@ RigActiveCellInfo* RimReservoirGridEnsemble::unionOfActiveCells( RiaDefines::Por
}
}
//--------------------------------------------------------------------------------------------------
/// Open every realization. Realizations can be opened lazily, and opening one can mean downloading a
/// grid, so this is only called by the operations that genuinely read them all.
//--------------------------------------------------------------------------------------------------
void RimReservoirGridEnsemble::ensureAllCasesAreOpen()
{
auto allCases = cases();
std::vector<RimEclipseCase*> casesToOpen;
for ( auto* eclipseCase : allCases )
{
if ( eclipseCase && !eclipseCase->eclipseCaseData() ) casesToOpen.push_back( eclipseCase );
}
if ( casesToOpen.empty() ) return;
caf::ProgressInfo progInfo( casesToOpen.size(), "Loading realizations" );
for ( auto* eclipseCase : casesToOpen )
{
eclipseCase->openReservoirCase();
progInfo.incrementProgress();
}
// More realizations contribute now, so any union computed earlier is incomplete.
clearActiveCellUnions();
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
@@ -383,7 +456,26 @@ void RimReservoirGridEnsemble::computeUnionOfActiveCells()
return;
}
if ( m_caseCollection->reservoirs.empty() || !m_mainGrid )
// The union is only correct when every realization has been read, and the result is cached by the
// check above. Realizations can be opened lazily, so make sure they are all open first.
ensureAllCasesAreOpen();
if ( m_caseCollection->reservoirs.empty() || m_mainGrid.isNull() )
{
clearActiveCellUnions();
return;
}
// Only realizations that are open and actually use the shared grid can contribute: a realization
// whose dimensions did not match kept a grid of its own, and its cell indices mean nothing here.
std::vector<RigEclipseCaseData*> contributingCases;
for ( auto* eclipseCase : m_caseCollection->reservoirs.childrenByType() )
{
auto* caseData = eclipseCase ? eclipseCase->eclipseCaseData() : nullptr;
if ( caseData && caseData->mainGrid() == m_mainGrid.p() ) contributingCases.push_back( caseData );
}
if ( contributingCases.empty() )
{
clearActiveCellUnions();
return;
@@ -404,32 +496,25 @@ void RimReservoirGridEnsemble::computeUnionOfActiveCells()
std::vector<char> activeM( grid->cellCount(), 0 );
std::vector<char> activeF( grid->cellCount(), 0 );
for ( size_t gridLocalCellIndex = 0; gridLocalCellIndex < grid->cellCount(); gridLocalCellIndex++ )
for ( auto* caseData : contributingCases )
{
for ( size_t caseIdx = 0; caseIdx < m_caseCollection->reservoirs.size(); caseIdx++ )
const auto* matrixActiveCells = caseData->activeCellInfo( RiaDefines::PorosityModelType::MATRIX_MODEL );
const auto* fractureActiveCells = caseData->activeCellInfo( RiaDefines::PorosityModelType::FRACTURE_MODEL );
for ( size_t gridLocalCellIndex = 0; gridLocalCellIndex < grid->cellCount(); gridLocalCellIndex++ )
{
size_t reservoirCellIndex = grid->reservoirCellIndex( gridLocalCellIndex );
if ( activeM[gridLocalCellIndex] == 0 )
if ( activeM[gridLocalCellIndex] == 0 && matrixActiveCells &&
matrixActiveCells->isActive( ReservoirCellIndex( reservoirCellIndex ) ) )
{
if ( m_caseCollection->reservoirs[caseIdx]
->eclipseCaseData()
->activeCellInfo( RiaDefines::PorosityModelType::MATRIX_MODEL )
->isActive( ReservoirCellIndex( reservoirCellIndex ) ) )
{
activeM[gridLocalCellIndex] = 1;
}
activeM[gridLocalCellIndex] = 1;
}
if ( activeF[gridLocalCellIndex] == 0 )
if ( activeF[gridLocalCellIndex] == 0 && fractureActiveCells &&
fractureActiveCells->isActive( ReservoirCellIndex( reservoirCellIndex ) ) )
{
if ( m_caseCollection->reservoirs[caseIdx]
->eclipseCaseData()
->activeCellInfo( RiaDefines::PorosityModelType::FRACTURE_MODEL )
->isActive( ReservoirCellIndex( reservoirCellIndex ) ) )
{
activeF[gridLocalCellIndex] = 1;
}
activeF[gridLocalCellIndex] = 1;
}
}
}
@@ -635,6 +720,14 @@ void RimReservoirGridEnsemble::createGridCasesFromEnsembleFileSet()
{
if ( !m_ensembleFileSet ) return;
recreateCaseObjects();
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RimReservoirGridEnsemble::recreateCaseObjects()
{
// Clear existing cases and statistics
m_caseCollection->reservoirs.deleteChildren();
m_statisticsCaseCollection->reservoirs.deleteChildren();
@@ -642,12 +735,20 @@ void RimReservoirGridEnsemble::createGridCasesFromEnsembleFileSet()
clearActiveCellUnions();
// Create case objects without loading grids
createCaseObjectsFromEnsembleFileSet();
createCaseObjects();
updateDataFilterCollectionCase();
updateConnectedEditors();
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
RimCaseCollection* RimReservoirGridEnsemble::caseCollection() const
{
return m_caseCollection;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
@@ -733,7 +834,10 @@ void RimReservoirGridEnsemble::defineUiOrdering( QString uiConfigName, caf::PdmU
{
uiOrdering.add( nameField() );
uiOrdering.add( &m_groupId );
uiOrdering.add( &m_ensembleFileSet );
// A subclass can take the realizations from somewhere else than a file set, see RimReservoirGridEnsembleSumo.
if ( m_ensembleFileSet ) uiOrdering.add( &m_ensembleFileSet );
uiOrdering.add( &m_autoDetectGridType );
if ( m_autoDetectGridType )
@@ -813,7 +917,7 @@ void RimReservoirGridEnsemble::fieldChangedByUi( const caf::PdmFieldHandle* chan
//--------------------------------------------------------------------------------------------------
void RimReservoirGridEnsemble::createDerivedObjects()
{
createCaseObjectsFromEnsembleFileSet();
createCaseObjects();
}
//--------------------------------------------------------------------------------------------------
@@ -920,7 +1024,7 @@ void RimReservoirGridEnsemble::updateMainGridAndActiveCellsForStatisticsCases()
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RimReservoirGridEnsemble::createCaseObjectsFromEnsembleFileSet()
void RimReservoirGridEnsemble::createCaseObjects()
{
if ( !m_ensembleFileSet ) return;
@@ -956,7 +1060,7 @@ void RimReservoirGridEnsemble::createCaseObjectsFromEnsembleFileSet()
void RimReservoirGridEnsemble::loadGridDataFromFiles()
{
// Guard: Only load once
if ( m_mainGrid != nullptr ) return;
if ( m_mainGrid.notNull() ) return;
auto allCases = cases();
if ( allCases.empty() ) return;
@@ -1004,7 +1108,9 @@ void RimReservoirGridEnsemble::updateGridModeToolTip()
{
const bool individualGrids = ( gridMode() == GridModeType::INDIVIDUAL_GRIDS );
uiCapability()->setUiToolTip( individualGrids ? QString( "This ensemble has grids with varying number of K layers." ) : QString() );
uiCapability()->setUiToolTip( individualGrids ? QString( "The realizations of this ensemble have grids with differing dimensions, "
"so each realization keeps its own grid." )
: QString() );
}
//--------------------------------------------------------------------------------------------------
@@ -1078,7 +1184,7 @@ void RimReservoirGridEnsemble::loadGridsInSharedMode()
// Reading of active cell data can fail, and no case data is created for the case
if ( auto caseData = eclipseCase->eclipseCaseData() )
{
caseData->setMainGrid( m_mainGrid );
caseData->setMainGrid( m_mainGrid.p() );
}
else
{
@@ -60,6 +60,7 @@ class RimReservoirGridEnsemble : public RimNamedObject, public RimReservoirGridE
public:
RimReservoirGridEnsemble();
~RimReservoirGridEnsemble() override;
// Ensemble file set connection
void setEnsembleFileSet( RimEnsembleFileSet* ensembleFileSet );
@@ -70,8 +71,8 @@ public:
void setGroupId( int id );
// Case management
void addCase( RimEclipseCase* reservoir );
void removeCase( RimEclipseCase* reservoir );
void addCase( RimEclipseCase* reservoir ) override;
void removeCase( RimEclipseCase* reservoir ) override;
bool contains( RimEclipseCase* reservoir ) const;
std::vector<RimEclipseCase*> cases() const;
RimEclipseCase* mainCase() override;
@@ -80,8 +81,13 @@ public:
// Grid detection and shared grid
RigMainGrid* mainGrid() override;
RigMainGrid* shareOrAdoptMainGrid( RigMainGrid* candidate ) override;
void setupSharedGrid();
// Open every realization. Needed by the operations that read them all, as realizations can otherwise
// be opened lazily, see RimReservoirGridEnsembleSumo.
void ensureAllCasesAreOpen();
// Deferred loading control
void loadGridDataFromFiles();
bool isGridDataLoaded() const;
@@ -111,7 +117,7 @@ public:
RimEclipseView* addViewForCase( RimEclipseCase* eclipseCase );
std::vector<RimEclipseView*> allViews() const;
std::set<RimEclipseCase*> casesInViews() const override;
RimEclipseViewCollection* viewCollection() const;
RimEclipseViewCollection* viewCollection() const override;
// Well target mapping
void addWellTargetMapping( RimWellTargetMapping* wellTargetMapping );
@@ -136,6 +142,24 @@ protected:
QList<caf::PdmOptionItemInfo> calculateValueOptions( const caf::PdmFieldHandle* fieldNeedingOptions ) override;
// Drop the existing realization cases and rebuild them from the current source.
void recreateCaseObjects();
// Build the realization case objects, without loading the grids. The base implementation derives them
// from the ensemble file set; a subclass with another source of realizations overrides this.
virtual void createCaseObjects();
// Decide whether all realizations have the same grid dimensions. Overridden by a subclass that can
// answer this without reading the grids.
virtual bool detectGridDimensionEquality();
// Establish the grid shared by the ensemble. The base implementation opens every realization eagerly;
// a subclass where opening a realization is expensive can open only the first and leave the rest to be
// opened on demand.
virtual void loadGridsInSharedMode();
RimCaseCollection* caseCollection() const;
private:
void onFileSetChanged( const caf::SignalEmitter* emitter );
void clearActiveCellUnions();
@@ -143,13 +167,12 @@ private:
void updateMainGridAndActiveCellsForStatisticsCases();
void updateStatisticsVisibility();
void createCaseObjectsFromEnsembleFileSet();
bool detectGridDimensionEquality();
void loadGridsInSharedMode();
void loadGridsInIndividualMode();
void updateGridModeToolTip();
void updateDataFilterCollectionCase();
static bool hasMatchingDimensions( const RigMainGrid* lhs, const RigMainGrid* rhs );
private:
// File set reference
caf::PdmPtrField<RimEnsembleFileSet*> m_ensembleFileSet;
@@ -178,8 +201,9 @@ private:
caf::PdmChildArrayField<RimWellTargetMapping*> m_wellTargetMappings;
caf::PdmChildArrayField<RimStatisticsContourMap*> m_statisticsContourMaps;
// Shared grid data (for identical grids)
RigMainGrid* m_mainGrid;
// Shared grid data (for identical grids). Reference counted: with realizations opened lazily this can
// be the only owner of the grid, e.g. after the realization that loaded it has been deselected.
cvf::ref<RigMainGrid> m_mainGrid;
cvf::ref<RigActiveCellInfo> m_unionOfMatrixActiveCells;
cvf::ref<RigActiveCellInfo> m_unionOfFractureActiveCells;
};
@@ -30,6 +30,7 @@ class RigMainGrid;
class RimCaseCollection;
class RimEclipseCase;
class RimEclipseStatisticsCase;
class RimEclipseViewCollection;
class RimFormationNames;
class RimStatisticsContourMap;
@@ -49,6 +50,25 @@ public:
virtual std::vector<RimEclipseCase*> sourceCases() const = 0;
virtual RimEclipseCase* mainCase() = 0;
virtual void addCase( RimEclipseCase* reservoir ) = 0;
virtual void removeCase( RimEclipseCase* reservoir ) = 0;
// Null for ensembles that do not own a view collection.
virtual RimEclipseViewCollection* viewCollection() const { return nullptr; }
// Ensembles can opt out of the mobile volume weighted mean, as the data it is derived from can be
// expensive to fetch. Used by Sumo grid ensembles, see Rim3dOverlayInfoConfig.
virtual bool doComputeMobileVolumeWeightedMean() const { return true; }
virtual void setDoComputeMobileVolumeWeightedMean( bool enable ) {}
// Offer 'candidate' as the shared grid of this ensemble. Returns the grid the caller should use: the
// shared grid already in memory, when this ensemble shares grids and the dimensions match, otherwise
// 'candidate', which then becomes the shared grid.
//
// Never triggers grid loading, so it is safe to call while opening a case. Calling mainGrid() instead
// would re-enter the loading of the first realization and publish an empty grid.
virtual RigMainGrid* shareOrAdoptMainGrid( RigMainGrid* candidate ) { return candidate; }
virtual RigMainGrid* mainGrid();
virtual RigActiveCellInfo* unionOfActiveCells( RiaDefines::PorosityModelType porosityType );
virtual void computeUnionOfActiveCells();
@@ -0,0 +1,277 @@
/////////////////////////////////////////////////////////////////////////////////
//
// Copyright (C) 2026 Equinor ASA
//
// ResInsight is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// ResInsight is distributed in the hope that it will be useful, but WITHOUT ANY
// WARRANTY; without even the implied warranty of MERCHANTABILITY or
// FITNESS FOR A PARTICULAR PURPOSE.
//
// See the GNU General Public License at <http://www.gnu.org/licenses/gpl.html>
// for more details.
//
/////////////////////////////////////////////////////////////////////////////////
#include "RimReservoirGridEnsembleSumo.h"
#include "RiaLogging.h"
#include "RiaStdStringTools.h"
// The reference counted members of the base class need complete types here, so the destructor of this
// class can be generated.
#include "RigActiveCellInfo.h"
#include "RigMainGrid.h"
#include "Cloud/RimSumoDataSource.h"
#include "RimCaseCollection.h"
#include "RimEclipseCase.h"
#include "RimProject.h"
#include "RimRoffCaseSumo.h"
#include "cafPdmUiTreeAttributes.h"
#include <algorithm>
#include <format>
CAF_PDM_SOURCE_INIT( RimReservoirGridEnsembleSumo, "RimReservoirGridEnsembleSumo" );
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
RimReservoirGridEnsembleSumo::RimReservoirGridEnsembleSumo()
{
CAF_PDM_InitObject( "Sumo Reservoir Grid Ensemble", ":/GridCaseGroup16x16.png" );
CAF_PDM_InitFieldNoDefault( &m_sumoDataSource, "SumoDataSource", "Sumo Data Source" );
m_sumoDataSource.uiCapability()->setUiHidden( true );
CAF_PDM_InitFieldNoDefault( &m_gridName, "GridName", "Grid Name" );
m_gridName.uiCapability()->setUiReadOnly( true );
CAF_PDM_InitFieldNoDefault( &m_realizations, "Realizations", "Realizations" );
m_realizations.uiCapability()->setUiHidden( true );
CAF_PDM_InitFieldNoDefault( &m_gridRealizations, "GridRealizations", "Grid Realizations" );
m_gridRealizations.uiCapability()->setUiHidden( true );
CAF_PDM_InitFieldNoDefault( &m_missingGridDataInfo, "MissingGridDataInfo", "Grid Data" );
m_missingGridDataInfo.registerGetMethod( this, &RimReservoirGridEnsembleSumo::missingGridDataText );
m_missingGridDataInfo.uiCapability()->setUiReadOnly( true );
CAF_PDM_InitField( &m_gridDimensionsAreIdentical, "GridDimensionsAreIdentical", false, "Grid Dimensions Are Identical" );
m_gridDimensionsAreIdentical.uiCapability()->setUiHidden( true );
// Set by the data source creating the ensemble, see RimSumoDataSource.
CAF_PDM_InitField( &m_doComputeMobileVolumeWeightedMean, "DoComputeMobileVolumeWeightedMean", false, "Compute Mobile Volume Weighted Mean" );
m_doComputeMobileVolumeWeightedMean.uiCapability()->setUiHidden( true );
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RimReservoirGridEnsembleSumo::setSumoSource( RimSumoDataSource* dataSource,
const QString& gridName,
const std::vector<int>& realizations,
bool gridDimensionsAreIdentical )
{
m_sumoDataSource = dataSource;
m_gridName = gridName;
m_realizations = realizations;
m_gridDimensionsAreIdentical = gridDimensionsAreIdentical;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RimReservoirGridEnsembleSumo::setGridRealizations( const std::vector<int>& realizations )
{
m_gridRealizations = realizations;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
std::vector<int> RimReservoirGridEnsembleSumo::gridRealizations() const
{
return m_gridRealizations();
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
std::vector<int> RimReservoirGridEnsembleSumo::realizationsWithoutGridData() const
{
// An empty list of grid realizations means they are not known, and then nothing can be reported as
// missing. RimSumoDataSource skips the same filtering in that situation.
if ( !m_sumoDataSource || m_gridRealizations().empty() ) return {};
const auto gridRealizations = m_gridRealizations();
std::vector<int> missing;
for ( const QString& realizationId : m_sumoDataSource->selectedRealizationIds() )
{
bool ok = false;
int realization = realizationId.toInt( &ok );
if ( !ok ) continue;
if ( std::ranges::find( gridRealizations, realization ) == gridRealizations.end() ) missing.push_back( realization );
}
std::ranges::sort( missing );
return missing;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
QString RimReservoirGridEnsembleSumo::missingGridDataText() const
{
if ( !m_sumoDataSource ) return "No data source.";
if ( m_gridRealizations().empty() ) return "The realizations of the grid are not known.";
const auto missing = realizationsWithoutGridData();
if ( missing.empty() ) return "The grid exists for all realizations of the ensemble.";
return QString( "No grid data for %1 of the realizations of the ensemble: %2" )
.arg( missing.size() )
.arg( QString::fromStdString( RiaStdStringTools::formatRangeSelection( missing ) ) );
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RimReservoirGridEnsembleSumo::defineUiOrdering( QString uiConfigName, caf::PdmUiOrdering& uiOrdering )
{
RimReservoirGridEnsemble::defineUiOrdering( uiConfigName, uiOrdering );
uiOrdering.add( &m_missingGridDataInfo );
}
//--------------------------------------------------------------------------------------------------
/// Flag an ensemble that does not cover all its realizations, so the missing ones are visible in the
/// tree and not only in the property panel.
//--------------------------------------------------------------------------------------------------
void RimReservoirGridEnsembleSumo::defineObjectEditorAttribute( QString uiConfigName, caf::PdmUiEditorAttribute* attribute )
{
RimReservoirGridEnsemble::defineObjectEditorAttribute( uiConfigName, attribute );
if ( realizationsWithoutGridData().empty() ) return;
if ( auto* treeItemAttribute = dynamic_cast<caf::PdmUiTreeViewItemAttribute*>( attribute ) )
{
auto tag = caf::PdmUiTreeViewItemAttribute::createTag();
tag->icon = caf::IconProvider( ":/warning.svg" );
treeItemAttribute->tags.push_back( std::move( tag ) );
}
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
QString RimReservoirGridEnsembleSumo::sumoGridName() const
{
return m_gridName();
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
bool RimReservoirGridEnsembleSumo::doComputeMobileVolumeWeightedMean() const
{
return m_doComputeMobileVolumeWeightedMean();
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RimReservoirGridEnsembleSumo::setDoComputeMobileVolumeWeightedMean( bool enable )
{
m_doComputeMobileVolumeWeightedMean = enable;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RimReservoirGridEnsembleSumo::createGridCasesFromSumoSource()
{
if ( !m_sumoDataSource ) return;
recreateCaseObjects();
}
//--------------------------------------------------------------------------------------------------
/// Create one grid case per realization. Called both when the ensemble is created and when the project is
/// loaded, as the realization cases are not written to the project file.
//--------------------------------------------------------------------------------------------------
void RimReservoirGridEnsembleSumo::createCaseObjects()
{
if ( !m_sumoDataSource ) return;
auto* project = RimProject::current();
for ( int realization : m_realizations() )
{
auto* gridCase = RimRoffCaseSumo::createFromDataSource( m_sumoDataSource, m_gridName(), realization );
if ( !gridCase ) continue;
// The ensemble is put in the project tree before the cases are created, so the ids are unique.
if ( project ) project->assignCaseIdToCase( gridCase );
caseCollection()->reservoirs().push_back( gridCase );
}
}
//--------------------------------------------------------------------------------------------------
/// Sumo reports the IJK dimensions of every realization, so this is decided when the ensemble is created,
/// without reading any grid. See RicCreateSumoReservoirGridEnsembleFeature.
//--------------------------------------------------------------------------------------------------
bool RimReservoirGridEnsembleSumo::detectGridDimensionEquality()
{
return m_gridDimensionsAreIdentical();
}
//--------------------------------------------------------------------------------------------------
/// Open the first realization, whose grid becomes the grid shared by the whole ensemble. The other
/// realizations are opened lazily, each downloading and parsing its own blob to read its own active
/// cells before releasing the parsed geometry, see RimRoffCaseSumo::openEclipseGridFile. Opening a
/// realization means downloading a grid, so they are not opened up front.
//--------------------------------------------------------------------------------------------------
void RimReservoirGridEnsembleSumo::loadGridsInSharedMode()
{
auto allCases = cases();
if ( allCases.empty() ) return;
RiaLogging::info(
std::format( "Grid ensemble '{}': loading the shared grid from the first of {} realizations.", name().toStdString(), allCases.size() ) );
// openReservoirCase, not openEclipseGridFile, so the characteristic cell size and the face normals are
// computed once, on the grid the other realizations share.
allCases[0]->openReservoirCase();
// computeUnionOfActiveCells is deliberately not called here: only one realization is open, and the
// union caches its result. It is computed on demand, after every realization has been opened.
}
//--------------------------------------------------------------------------------------------------
/// The realization cases are not written to the project file, they are recreated by createCaseObjects.
/// Capture the realizations in the order the cases appear, so they come back in the same order: the
/// references from the views to the cases are positional.
//--------------------------------------------------------------------------------------------------
void RimReservoirGridEnsembleSumo::setupBeforeSave()
{
std::vector<int> realizations;
for ( auto* eclipseCase : cases() )
{
if ( auto* sumoCase = dynamic_cast<RimRoffCaseSumo*>( eclipseCase ) )
{
realizations.push_back( sumoCase->realization() );
}
}
m_realizations = realizations;
}
@@ -0,0 +1,106 @@
/////////////////////////////////////////////////////////////////////////////////
//
// Copyright (C) 2026 Equinor ASA
//
// ResInsight is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// ResInsight is distributed in the hope that it will be useful, but WITHOUT ANY
// WARRANTY; without even the implied warranty of MERCHANTABILITY or
// FITNESS FOR A PARTICULAR PURPOSE.
//
// See the GNU General Public License at <http://www.gnu.org/licenses/gpl.html>
// for more details.
//
/////////////////////////////////////////////////////////////////////////////////
#pragma once
#include "RimReservoirGridEnsemble.h"
#include "cafPdmField.h"
#include "cafPdmProxyValueField.h"
#include "cafPdmPtrField.h"
#include <QString>
#include <vector>
class RimSumoDataSource;
//==================================================================================================
///
/// RimReservoirGridEnsembleSumo - Grid ensemble of one grid stored on Sumo, one case per realization
///
/// The realizations come from a RimSumoDataSource instead of an ensemble file set. As in the base
/// class the realization cases are not written to the project file, they are recreated from the data
/// source when the project is loaded, see createCaseObjects.
///
/// When all realizations report the same IJK dimensions, one RigMainGrid is kept in memory for the
/// whole ensemble. Every realization still downloads and parses its own grid blob to read its own
/// active cells, then releases the parsed geometry and points at the shared grid, see
/// RimRoffCaseSumo::openEclipseGridFile. Realizations are opened lazily, the first time one is
/// displayed; only the first realization is loaded when the ensemble is created.
///
//==================================================================================================
class RimReservoirGridEnsembleSumo : public RimReservoirGridEnsemble
{
CAF_PDM_HEADER_INIT;
public:
RimReservoirGridEnsembleSumo();
// gridDimensionsAreIdentical decides whether the realizations can share one grid. It comes from the
// dimensions reported by Sumo, so no grid has to be downloaded to decide.
void setSumoSource( RimSumoDataSource* dataSource,
const QString& gridName,
const std::vector<int>& realizations,
bool gridDimensionsAreIdentical );
// The realizations this grid exists for, which is not necessarily every realization of the ensemble.
void setGridRealizations( const std::vector<int>& realizations );
std::vector<int> gridRealizations() const;
// The realizations of the data source this grid has no data for, sorted. The data source decides which
// realizations the ensemble has, without regard for which data types exist for them, so a grid can be
// missing for some of them. Computed on demand: the realization selection can change after the ensemble
// is created, see RimSumoDataSource.
std::vector<int> realizationsWithoutGridData() const;
QString sumoGridName() const;
void createGridCasesFromSumoSource();
bool doComputeMobileVolumeWeightedMean() const override;
void setDoComputeMobileVolumeWeightedMean( bool enable ) override;
protected:
void defineUiOrdering( QString uiConfigName, caf::PdmUiOrdering& uiOrdering ) override;
void defineObjectEditorAttribute( QString uiConfigName, caf::PdmUiEditorAttribute* attribute ) override;
void createCaseObjects() override;
bool detectGridDimensionEquality() override;
void loadGridsInSharedMode() override;
void setupBeforeSave() override;
private:
QString missingGridDataText() const;
private:
caf::PdmPtrField<RimSumoDataSource*> m_sumoDataSource;
caf::PdmField<QString> m_gridName;
// The realizations to create cases for, in the order the cases appear. The cases themselves are not
// written to the project file, see the class comment.
caf::PdmField<std::vector<int>> m_realizations;
caf::PdmField<std::vector<int>> m_gridRealizations;
// Reports the realizations without grid data. A proxy field, so it follows a changed realization
// selection instead of going stale.
caf::PdmProxyValueField<QString> m_missingGridDataInfo;
caf::PdmField<bool> m_gridDimensionsAreIdentical;
caf::PdmField<bool> m_doComputeMobileVolumeWeightedMean;
};
@@ -0,0 +1,744 @@
/////////////////////////////////////////////////////////////////////////////////
//
// Copyright (C) 2024 Equinor ASA
//
// ResInsight is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// ResInsight is distributed in the hope that it will be useful, but WITHOUT ANY
// WARRANTY; without even the implied warranty of MERCHANTABILITY or
// FITNESS FOR A PARTICULAR PURPOSE.
//
// See the GNU General Public License at <http://www.gnu.org/licenses/gpl.html>
// for more details.
//
/////////////////////////////////////////////////////////////////////////////////
#include "RimRoffCaseSumo.h"
#include "RiaApplication.h"
#include "RiaLogging.h"
#include "RiaPreferencesGrid.h"
#include "RiaResultNames.h"
#include "Cloud/RiaSumoConnector.h"
#include "Cloud/RimSumoDataSource.h"
#include "RifReaderSumoGridProperty.h"
#include "RifRoffFileTools.h"
#include "RigActiveCellInfo.h"
#include "RigCaseCellResultsData.h"
#include "RigEclipseCaseData.h"
#include "RigEclipseResultAddress.h"
#include "RigEclipseResultInfo.h"
#include "RigMainGrid.h"
#include "RimCaseCollection.h"
#include "RimEclipseCellColors.h"
#include "RimEclipseView.h"
#include "RimReservoirCellResultsStorage.h"
#include "RimReservoirGridEnsembleBase.h"
#include "RiuMainWindow.h"
#include "RiuViewer.h"
#include "cafPdmObjectScriptingCapability.h"
#include "cafPdmPointer.h"
#include "cafProgressInfo.h"
#include <QDate>
#include <QDateTime>
#include <QStatusBar>
#include <algorithm>
#include <map>
#include <set>
#include <sstream>
#include <vector>
CAF_PDM_SOURCE_INIT( RimRoffCaseSumo, "RimRoffCaseSumo" );
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
RimRoffCaseSumo::RimRoffCaseSumo()
{
CAF_PDM_InitScriptableObject( "Sumo Grid Case", ":/Case48x48.png" );
CAF_PDM_InitFieldNoDefault( &m_sumoDataSource, "SumoDataSource", "Sumo Data Source" );
m_sumoDataSource.uiCapability()->setUiHidden( true );
CAF_PDM_InitFieldNoDefault( &m_sumoCaseId, "SumoCaseId", "Sumo Case Id" );
m_sumoCaseId.uiCapability()->setUiReadOnly( true );
CAF_PDM_InitFieldNoDefault( &m_ensembleName, "EnsembleName", "Ensemble Name" );
m_ensembleName.uiCapability()->setUiReadOnly( true );
CAF_PDM_InitFieldNoDefault( &m_gridName, "GridName", "Grid Name" );
m_gridName.uiCapability()->setUiReadOnly( true );
CAF_PDM_InitField( &m_realization, "Realization", -1, "Realization" );
m_realization.uiCapability()->setUiReadOnly( true );
m_sumoConnector = RiaApplication::instance()->makeSumoConnector();
m_lifetimeToken = std::make_shared<bool>( true );
}
//--------------------------------------------------------------------------------------------------
/// Aborts any transfers this case still has in flight, see RiaSumoConnector::cancelGroup.
///
/// Rotate m_lifetimeToken before calling cancelGroup(), not after: an abort can make Qt deliver a
/// reply's finished() synchronously, re-entering this case's own completion lambdas while this
/// destructor is still on the stack. Those lambdas key off m_lifetimeToken.get() only to identify
/// which cancelGroup() call this transfer belongs to, not for an aliveness check (they rely on a
/// caf::PdmPointer for that) - but see RifReaderSumoGridProperty::~RifReaderSumoGridProperty() for
/// why this ordering still matters wherever a lifetime token is involved. Reset first, capture the
/// old raw pointer for the call, matching that fix for consistency/defense in depth.
//--------------------------------------------------------------------------------------------------
RimRoffCaseSumo::~RimRoffCaseSumo()
{
void* lifetimeTokenKey = m_lifetimeToken.get();
m_lifetimeToken.reset();
if ( m_sumoConnector ) m_sumoConnector->cancelGroup( lifetimeTokenKey );
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
RimRoffCaseSumo* RimRoffCaseSumo::createFromDataSource( RimSumoDataSource* dataSource, const QString& gridName, int realization )
{
if ( !dataSource ) return nullptr;
auto* gridCase = new RimRoffCaseSumo();
gridCase->setSumoDataSource( dataSource );
gridCase->setSumoCaseId( dataSource->caseId().get() );
gridCase->setEnsembleName( dataSource->ensembleName() );
gridCase->setGridName( gridName );
gridCase->setRealization( realization );
// The grid is stored on Sumo, not on disk, so there is no real grid file name. Still assign a unique
// synthetic one: RimReservoirGridEnsemble identifies its realizations by grid file name, see contains()
// and findByFileName().
gridCase->setGridFileName(
QString( "sumo/%1/%2/realization-%3/%4.roff" ).arg( dataSource->caseId().get() ).arg( gridName ).arg( realization ).arg( gridName ) );
// Name the case using grid name, asset, ensemble and realization, e.g. "Geogrid_Drogon_iter-0_Real_0".
QString caseDisplayName =
QString( "%1_%2_%3_Real_%4" ).arg( gridName, dataSource->assetName(), dataSource->ensembleName() ).arg( realization );
gridCase->setCustomCaseName( caseDisplayName );
return gridCase;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RimRoffCaseSumo::setSumoDataSource( RimSumoDataSource* dataSource )
{
m_sumoDataSource = dataSource;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RimRoffCaseSumo::setSumoCaseId( const QString& sumoCaseId )
{
m_sumoCaseId = sumoCaseId;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RimRoffCaseSumo::setEnsembleName( const QString& ensembleName )
{
m_ensembleName = ensembleName;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RimRoffCaseSumo::setGridName( const QString& gridName )
{
m_gridName = gridName;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RimRoffCaseSumo::setRealization( int realization )
{
m_realization = realization;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
QString RimRoffCaseSumo::gridName() const
{
return m_gridName();
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
int RimRoffCaseSumo::realization() const
{
return m_realization();
}
//--------------------------------------------------------------------------------------------------
/// Every realization downloads and parses its own grid blob, because a roff grid carries its geometry and
/// its active cells in one stream and there is no way to read the active cells alone. When the ensemble
/// already holds a grid of the same dimensions, the geometry just parsed is released and the shared grid
/// is used instead: shared grid mode keeps one RigMainGrid in memory for the whole ensemble rather than
/// one per realization. The active cells are always this realization's own, ACTNUM is never assumed to be
/// equal across realizations.
//--------------------------------------------------------------------------------------------------
bool RimRoffCaseSumo::openEclipseGridFile()
{
if ( eclipseCaseData() )
{
// Early exit if reservoir data is created
return true;
}
if ( !m_sumoConnector )
{
RiaLogging::error( "No Sumo connector available, unable to load grid from Sumo." );
return false;
}
setReservoirData( new RigEclipseCaseData( this ) );
// Before the grid, not after: the property transfer needs no grid to start, so the two run in parallel
// instead of one after the other.
startPropertyFetch();
// RifRoffFileTools::openGridFile appends into the grid it is given, so this must be a grid of our own.
// The shared grid can only be adopted once the parsing is done.
if ( !downloadAndParseGrid() )
{
// Leave no half built case behind, so opening this case can be retried after a failed transfer.
setReservoirData( nullptr );
return false;
}
RigMainGrid* parsedGrid = eclipseCaseData()->mainGrid();
RigMainGrid* gridToUse = parsedGrid;
QString gridOwnership = "own grid (case is not in a reservoir grid ensemble)";
RimReservoirGridEnsembleBase* gridEnsemble = parentCaseCollection() ? parentCaseCollection()->parentGridEnsembleBase() : nullptr;
if ( gridEnsemble )
{
gridToUse = gridEnsemble->shareOrAdoptMainGrid( parsedGrid );
gridOwnership = ( gridToUse != parsedGrid ) ? "shared grid"
: gridEnsemble->gridMode() == RimReservoirGridEnsembleBase::GridModeType::SHARED_GRID
? "own grid (published as the ensemble shared grid)"
: "own grid (ensemble uses individual grids)";
}
if ( gridToUse != parsedGrid )
{
// Point at the shared grid and release the geometry just parsed. The active cell info produced by
// the same parse belongs to this realization and is kept.
eclipseCaseData()->setMainGrid( gridToUse );
// The cell search tree, the bounding box and the faults of the shared grid were computed by the
// realization that loaded it, so computeCachedData() is not repeated here. The active cell bounding
// box is per realization and must be computed, as in RimEclipseResultCase::openAndReadActiveCellData.
computeActiveCellsBoundingBox();
eclipseCaseData()->setActiveFormationNames( effectiveFormationNames() );
}
else
{
parsedGrid->setFlipAxis( m_flipXAxis, m_flipYAxis );
computeCachedData();
}
// The same grid address for every realization, with the reference count climbing, is what shows the
// grid is shared. The active cell count is this realization's own, and is expected to vary between
// realizations even when they share a grid.
const auto* activeCells = eclipseCaseData()->activeCellInfo( RiaDefines::PorosityModelType::MATRIX_MODEL );
RiaLogging::debug( std::format( "Sumo grid '{}' realization {}: {} {}, {}x{}x{}, {} active cells, grid ref count {}",
m_gridName().toStdString(),
m_realization(),
gridOwnership.toStdString(),
static_cast<const void*>( gridToUse ),
gridToUse->cellCountI(),
gridToUse->cellCountJ(),
gridToUse->cellCountK(),
activeCells ? activeCells->reservoirActiveCellCount() : 0,
gridToUse->refCount() ) );
finalizeCaseSetup();
return true;
}
//--------------------------------------------------------------------------------------------------
/// The case can be closed and reopened many times while this object stays alive, e.g. when statistics for an
/// ensemble contour map release each realization's memory after processing it. m_propertyReader is a second
/// reference to the reader alongside the one held by the cell results: releasing it here is what lets the
/// reader (and its lifetime token) actually be destroyed when the case data is, instead of surviving with a
/// dangling m_caseData pointer that a Sumo transfer arriving later would write into.
//--------------------------------------------------------------------------------------------------
void RimRoffCaseSumo::closeReservoirCase()
{
m_propertyReader = nullptr;
cancelPendingTransfers();
RimEclipseCase::closeReservoirCase();
}
//--------------------------------------------------------------------------------------------------
/// Aborts the speculative fetch startPropertyFetch issues before the reader exists, see its cancelGroup
/// argument below. Without this, closing (or switching away from) this realization while that transfer is
/// still on its way leaves it running to completion, consuming a connection and bandwidth that a
/// newly selected realization's grid download needs. Unlike closeReservoirCase, this keeps already loaded
/// grid and result data intact, so switching back to this realization does not force a full reload.
//--------------------------------------------------------------------------------------------------
void RimRoffCaseSumo::cancelPendingTransfers()
{
m_fetchInFlight.reset();
// Rotate the token before cancelling, not after: see ~RimRoffCaseSumo for why a reentrant delivery
// during cancelGroup() must observe an already-superseded token, not the one about to be cancelled.
void* oldLifetimeTokenKey = m_lifetimeToken.get();
m_lifetimeToken = std::make_shared<bool>( true );
if ( m_sumoConnector ) m_sumoConnector->cancelGroup( oldLifetimeTokenKey );
}
//--------------------------------------------------------------------------------------------------
/// See header and RifReaderSumoGridProperty::prefetchDynamicResult.
//--------------------------------------------------------------------------------------------------
bool RimRoffCaseSumo::prefetchDynamicResult( const QString& resultName, const std::vector<size_t>& stepIndices )
{
if ( m_propertyReader.isNull() ) return false;
return m_propertyReader->prefetchDynamicResult( resultName, stepIndices );
}
//--------------------------------------------------------------------------------------------------
/// Download the roff grid blob for this realization and parse it into the case data.
//--------------------------------------------------------------------------------------------------
bool RimRoffCaseSumo::downloadAndParseGrid()
{
QByteArray contents = m_sumoConnector->grid().gridData( SumoCaseId( m_sumoCaseId() ), m_ensembleName(), m_gridName(), m_realization() );
if ( contents.isEmpty() )
{
RiaLogging::error(
std::format( "Failed to download grid '{}' (realization {}) from Sumo.", m_gridName().toStdString(), m_realization() ) );
return false;
}
// The downloaded blob is a binary roff grid. Parse it directly from memory.
std::string buffer = contents.toStdString();
std::istringstream stream( buffer, std::ios::binary );
QString errorMessages;
if ( !RifRoffFileTools::openGridFile( stream, eclipseCaseData(), &errorMessages ) )
{
RiaLogging::error( errorMessages.toStdString() );
return false;
}
return true;
}
//--------------------------------------------------------------------------------------------------
/// Register the results and attach the property reader. Shared by the two ways the grid can end up on
/// this case: parsed and kept, or parsed and replaced by the grid shared with the other realizations.
//--------------------------------------------------------------------------------------------------
void RimRoffCaseSumo::finalizeCaseSetup()
{
results( RiaDefines::PorosityModelType::MATRIX_MODEL )->createPlaceholderResultEntries();
if ( RiaPreferencesGrid::current()->autoComputeDepthRelatedProperties() )
{
eclipseCaseData()->computeDepthRelatedResults();
}
results( RiaDefines::PorosityModelType::MATRIX_MODEL )->computeCellVolumes();
// Make the Sumo grid properties available as cell results (fetched on demand when displayed).
registerSumoGridProperties();
// Rebuild the Data Sources result folders now that the result meta data is available.
updateResultAddressCollection();
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
QString RimRoffCaseSumo::locationOnDisc() const
{
// The grid is stored on Sumo, not on disk.
return QString();
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RimRoffCaseSumo::defineUiOrdering( QString uiConfigName, caf::PdmUiOrdering& uiOrdering )
{
uiOrdering.add( &m_caseUserDescription );
uiOrdering.add( &m_displayNameOption );
uiOrdering.add( &m_caseId );
auto group = uiOrdering.addNewGroup( "Sumo" );
group->add( &m_sumoCaseId );
group->add( &m_ensembleName );
group->add( &m_gridName );
group->add( &m_realization );
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RimRoffCaseSumo::registerSumoGridProperties()
{
if ( !m_sumoConnector || !eclipseCaseData() ) return;
// Already fetched by startPropertyFetch in the normal case. Only ask again when it is not there, so a
// case opened without that step still works.
if ( m_propertyInfos.empty() )
{
m_propertyInfos = m_sumoConnector->grid().propertyInfo( SumoCaseId( m_sumoCaseId() ), m_ensembleName(), m_gridName(), m_realization() );
}
const auto& gridPropertyInfos = m_propertyInfos;
// Properties without a timestamp are static. Properties with a single timestamp are dynamic (one time
// step per timestamp). Time intervals (the iso string contains '/') are not supported and skipped.
std::vector<QString> staticPropertyNames;
std::map<QString, std::set<QString>> dynamicPropertyTimestamps; // property name -> the timestamps it has
std::set<QString> allTimestamps; // union of timestamps across all properties
for ( const auto& info : gridPropertyInfos )
{
if ( info.isoDateOrInterval.isEmpty() )
{
staticPropertyNames.push_back( info.name );
}
else if ( !info.isoDateOrInterval.contains( '/' ) )
{
dynamicPropertyTimestamps[info.name].insert( info.isoDateOrInterval );
allTimestamps.insert( info.isoDateOrInterval );
}
}
if ( staticPropertyNames.empty() && dynamicPropertyTimestamps.empty() ) return;
// The Eclipse readers take the available phases from the INIT file. There is no INIT file here, so derive
// the phases from the saturation properties the case actually has. Without this the phase set stays empty,
// and RigCaseCellResultsData::defaultResult() skips its SOIL and SGAS branches - both are guarded by the
// phase set, while the SWAT branch is not - so a new view would always open on SWAT.
auto hasProperty = [&]( const QString& name )
{ return dynamicPropertyTimestamps.contains( name ) || std::ranges::find( staticPropertyNames, name ) != staticPropertyNames.end(); };
std::set<RiaDefines::PhaseType> availablePhases;
if ( hasProperty( RiaResultNames::soil() ) ) availablePhases.insert( RiaDefines::PhaseType::OIL_PHASE );
if ( hasProperty( RiaResultNames::sgas() ) ) availablePhases.insert( RiaDefines::PhaseType::GAS_PHASE );
if ( hasProperty( RiaResultNames::swat() ) ) availablePhases.insert( RiaDefines::PhaseType::WATER_PHASE );
if ( !availablePhases.empty() ) eclipseCaseData()->setAvailablePhases( availablePhases );
auto cellResults = results( RiaDefines::PorosityModelType::MATRIX_MODEL );
if ( !cellResults ) return;
auto reader = new RifReaderSumoGridProperty( m_sumoConnector, m_sumoCaseId(), m_ensembleName(), m_gridName(), m_realization() );
reader->open( "", eclipseCaseData() );
// Register the property names as cell results so they are listed in the cell result editor. The values are
// not loaded here; the reader fetches them on demand the first time a property is displayed.
for ( const auto& name : staticPropertyNames )
{
RigEclipseResultAddress resultAddress( RiaDefines::ResultCatType::STATIC_NATIVE, RiaDefines::ResultDataType::FLOAT, name );
cellResults->createResultEntry( resultAddress, false );
}
reader->setStaticProperties( staticPropertyNames );
if ( !dynamicPropertyTimestamps.empty() )
{
auto parseTimestamp = []( const QString& isoString ) -> QDateTime
{
QDateTime dateTime = QDateTime::fromString( isoString, Qt::ISODate );
if ( !dateTime.isValid() )
{
// Date-only string, e.g. "2018-01-01".
QDate date = QDate::fromString( isoString, Qt::ISODate );
if ( date.isValid() ) dateTime = QDateTime( date, QTime( 0, 0, 0 ) );
}
return dateTime;
};
// All dynamic results share one common, case-wide set of time steps so they use the same time step
// index space as the 3D view time slider. std::set is sorted, and ISO date strings sort chronologically.
std::vector<QString> commonTimestamps( allTimestamps.begin(), allTimestamps.end() );
std::vector<QDateTime> dates;
std::vector<int> reportNumbers;
std::vector<double> daysSinceStart;
for ( int i = 0; i < static_cast<int>( commonTimestamps.size() ); i++ )
{
QDateTime date = parseTimestamp( commonTimestamps[i] );
dates.push_back( date );
reportNumbers.push_back( i );
daysSinceStart.push_back( ( dates.front().isValid() && date.isValid() ) ? dates.front().daysTo( date ) : static_cast<double>( i ) );
}
auto commonTimeStepInfos = RigEclipseTimeStepInfo::createTimeStepInfos( dates, reportNumbers, daysSinceStart );
// For each property, build a list aligned with commonTimestamps. An empty entry marks a time step the
// property has no data for, so the reader reports "no data" there instead of another step's values.
std::map<QString, std::vector<QString>> readerDynamicTimestamps;
for ( const auto& [name, propertyTimestamps] : dynamicPropertyTimestamps )
{
RigEclipseResultAddress resultAddress( RiaDefines::ResultCatType::DYNAMIC_NATIVE, RiaDefines::ResultDataType::FLOAT, name );
cellResults->createResultEntry( resultAddress, false );
cellResults->setTimeStepInfos( resultAddress, commonTimeStepInfos );
std::vector<QString> alignedTimestamps;
alignedTimestamps.reserve( commonTimestamps.size() );
for ( const auto& timestamp : commonTimestamps )
{
alignedTimestamps.push_back( propertyTimestamps.count( timestamp ) > 0 ? timestamp : QString() );
}
readerDynamicTimestamps[name] = alignedTimestamps;
}
reader->setDynamicProperties( readerDynamicTimestamps );
}
m_propertyReader = reader;
reader->setPendingChangedCallback( [this]( const QString& message ) { onPropertyPendingChanged( message ); } );
reader->setTimeStepArrivedCallback( [this]( const QString& propertyName, size_t stepIndex, bool ok )
{ onPropertyTimeStepArrived( propertyName, stepIndex, ok ); } );
reader->setBatchProgressChangedCallback( [this]( size_t completed, size_t total, const QString& description )
{ onPropertyBatchProgressChanged( completed, total, description ); } );
// The transfer started before this reader existed is still on its way. Tell the reader, so it reports the
// wait to the user and does not issue a second request for the same time step.
if ( m_fetchInFlight ) reader->markTimeStepPending( m_fetchInFlight->first, m_fetchInFlight->second );
cellResults->setReaderInterface( reader );
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
QString RimRoffCaseSumo::dataLoadingText() const
{
if ( m_propertyReader.isNull() ) return {};
const QString pending = m_propertyReader->pendingDataDescription();
if ( pending.isEmpty() ) return {};
return QString( "Loading %1 from Sumo" ).arg( pending );
}
//--------------------------------------------------------------------------------------------------
/// The case wide time step list: the union of the timestamps of every dynamic property, sorted. All dynamic
/// results share it, so it is the index space the 3D view time slider works in.
//--------------------------------------------------------------------------------------------------
static std::vector<QString> commonTimestampsFromPropertyInfo( const std::vector<SumoGridPropertyInfo>& infos )
{
std::set<QString> allTimestamps;
for ( const auto& info : infos )
{
// Time intervals are not supported, and a static property has no timestamp at all.
if ( info.isoDateOrInterval.isEmpty() || info.isoDateOrInterval.contains( '/' ) ) continue;
allTimestamps.insert( info.isoDateOrInterval );
}
return { allTimestamps.begin(), allTimestamps.end() };
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
bool RimRoffCaseSumo::propertyToFetch( QString& propertyName, size_t& stepIndex, QString& isoDateOrInterval ) const
{
// What a view of this case is about to show. Without one there is nothing to guess at, and the on demand
// path will fetch whatever is asked for soon enough.
for ( auto* view : reservoirViews() )
{
if ( !view || !view->cellResult() ) continue;
const QString candidate = view->cellResult()->resultVariable();
if ( candidate.isEmpty() ) continue;
const int currentStep = view->currentTimeStep();
if ( currentStep < 0 ) continue;
const auto timestamps = commonTimestampsFromPropertyInfo( m_propertyInfos );
if ( static_cast<size_t>( currentStep ) >= timestamps.size() ) continue;
// Only worth fetching when this property actually has data at that time step.
const QString candidateTimestamp = timestamps[currentStep];
const bool hasThisTimeStep =
std::ranges::any_of( m_propertyInfos,
[&]( const auto& info ) { return info.name == candidate && info.isoDateOrInterval == candidateTimestamp; } );
if ( !hasThisTimeStep ) continue;
propertyName = candidate;
stepIndex = static_cast<size_t>( currentStep );
isoDateOrInterval = candidateTimestamp;
return true;
}
return false;
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RimRoffCaseSumo::startPropertyFetch()
{
if ( !m_sumoConnector ) return;
// Small request, and the only thing here that needs waiting for. It also spares the later registration a
// second one.
m_propertyInfos = m_sumoConnector->grid().propertyInfo( SumoCaseId( m_sumoCaseId() ), m_ensembleName(), m_gridName(), m_realization() );
QString propertyName;
size_t stepIndex = 0;
QString isoDateOrInterval;
if ( !propertyToFetch( propertyName, stepIndex, isoDateOrInterval ) ) return;
m_fetchInFlight = std::make_pair( propertyName, stepIndex );
RiaLogging::debug( std::format( "Fetching '{}' (time '{}') for realization {} in parallel with the grid.",
propertyName.toStdString(),
isoDateOrInterval.toStdString(),
m_realization() ) );
// Auto-nulls if this case is deleted while the transfer is in flight.
caf::PdmPointer<RimRoffCaseSumo> self( const_cast<RimRoffCaseSumo*>( this ) );
// Tied to m_lifetimeToken so closeReservoirCase can abort this transfer, e.g. when the view switches away
// from this realization while it is still on its way, see closeReservoirCase.
m_sumoConnector->grid().propertyDataBatchAsync(
SumoCaseId( m_sumoCaseId() ),
m_ensembleName(),
m_gridName(),
m_realization(),
propertyName,
{ isoDateOrInterval },
[self, propertyName, stepIndex]( const QString& iso, const QByteArray& contents )
{
if ( self.isNull() || contents.isEmpty() ) return;
// The reader is attached while the grid download holds
// this thread on a semaphore, which dispatches no
// events, so it is here by the time this runs. If it is
// somehow not, the values are dropped and the on demand
// path fetches them again.
self->m_fetchInFlight.reset();
if ( self->m_propertyReader.notNull() )
{
self->m_propertyReader->acceptFetchedTimeStep( propertyName, stepIndex, iso, contents );
}
},
m_lifetimeToken.get() );
}
//--------------------------------------------------------------------------------------------------
/// A banner in the view itself, and the status bar. The status bar is easy to miss and the info text is
/// small and can be switched off, while the wait is measured in seconds.
//--------------------------------------------------------------------------------------------------
void RimRoffCaseSumo::onPropertyPendingChanged( const QString& message )
{
for ( auto* view : reservoirViews() )
{
if ( !view || !view->viewer() ) continue;
view->viewer()->setLoadingText( message );
view->viewer()->showLoadingLabel( !message.isEmpty() );
}
auto* mainWindow = RiuMainWindow::instance();
if ( !mainWindow || !mainWindow->statusBar() ) return;
if ( message.isEmpty() )
{
mainWindow->statusBar()->clearMessage();
}
else
{
mainWindow->statusBar()->showMessage( message );
}
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RimRoffCaseSumo::onPropertyTimeStepArrived( const QString& /*propertyName*/, size_t /*stepIndex*/, bool ok )
{
// A failed transfer leaves its placeholder in place; nothing new to show, so no redraw.
if ( !ok ) return;
scheduleRedrawOfViews();
}
//--------------------------------------------------------------------------------------------------
/// Blocks the GUI with a single progress dialog for as long as the reader has something pending, instead of
/// one per time step waited for in RifReaderSumoGridProperty::waitForTimeStepToArrive. The reader itself only
/// reports plain completed/total/description numbers via this callback - no GUI code lives there.
//--------------------------------------------------------------------------------------------------
void RimRoffCaseSumo::onPropertyBatchProgressChanged( size_t completed, size_t total, const QString& description )
{
if ( total == 0 )
{
// Whole batch done (or nothing was ever pending): close the dialog rather than leaving a "100%" one
// lingering, and forget what it was constructed with so the next batch starts a fresh dialog.
m_batchProgress.reset();
m_batchProgressShownTotal = 0;
return;
}
if ( !m_batchProgress || m_batchProgressShownTotal != total )
{
// Either nothing is open yet, or the total grew while a dialog was already open (e.g. a redraw
// reached a time step outside the original request): caf::ProgressInfo's maximum is fixed at
// construction, so a growing total needs a fresh dialog rather than resizing the existing one.
const bool delayShowingProgress = false;
m_batchProgress = std::make_unique<caf::ProgressInfo>( total, "Downloading Sumo grid property data", delayShowingProgress );
m_batchProgressShownTotal = total;
}
m_batchProgress->setProgress( completed );
m_batchProgress->setProgressDescription( QString( "Downloaded %1 of %2 time steps - %3" ).arg( completed ).arg( total ).arg( description ) );
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RimRoffCaseSumo::scheduleRedrawOfViews()
{
// Redrawing reads cell results, which can start the next transfer and arrive back here through the
// reader's callback. Coalesce rather than recurse.
if ( m_isRedrawingViews )
{
m_hasMissedViewRedraw = true;
return;
}
m_isRedrawingViews = true;
do
{
m_hasMissedViewRedraw = false;
for ( auto* view : reservoirViews() )
{
if ( view ) view->scheduleCreateDisplayModelAndRedraw();
}
} while ( m_hasMissedViewRedraw );
m_isRedrawingViews = false;
}
@@ -0,0 +1,169 @@
/////////////////////////////////////////////////////////////////////////////////
//
// Copyright (C) 2024 Equinor ASA
//
// ResInsight is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// ResInsight is distributed in the hope that it will be useful, but WITHOUT ANY
// WARRANTY; without even the implied warranty of MERCHANTABILITY or
// FITNESS FOR A PARTICULAR PURPOSE.
//
// See the GNU General Public License at <http://www.gnu.org/licenses/gpl.html>
// for more details.
//
/////////////////////////////////////////////////////////////////////////////////
#pragma once
#include "RimEclipseCase.h"
#include "Cloud/RiaSumoGrid.h"
#include "cafPdmField.h"
#include "cafPdmPtrField.h"
#include "cvfObject.h"
#include <QPointer>
#include <QString>
#include <memory>
#include <optional>
#include <utility>
class RiaSumoConnector;
class RifReaderSumoGridProperty;
class RimSumoDataSource;
namespace caf
{
class ProgressInfo;
}
//==================================================================================================
//
// Eclipse grid case backed by a roff grid stored on Sumo. The grid geometry is downloaded as a
// blob through RiaSumoConnector and parsed in memory, so there is no grid file on disk.
//
//==================================================================================================
class RimRoffCaseSumo : public RimEclipseCase
{
CAF_PDM_HEADER_INIT;
public:
RimRoffCaseSumo();
~RimRoffCaseSumo() override;
// Create a grid case for a single realization of the given grid, linked back to the data source
// so the case can be updated when the data source realization filter changes.
static RimRoffCaseSumo* createFromDataSource( RimSumoDataSource* dataSource, const QString& gridName, int realization );
void setSumoDataSource( RimSumoDataSource* dataSource );
void setSumoCaseId( const QString& sumoCaseId );
void setEnsembleName( const QString& ensembleName );
void setGridName( const QString& gridName );
void setRealization( int realization );
QString gridName() const;
int realization() const;
bool openEclipseGridFile() override;
void closeReservoirCase() override;
// Aborts any transfers still in flight for this realization, without discarding already loaded grid and
// result data. Used when switching away from this case in a view while it may still be shown elsewhere,
// so a still loading realization does not keep competing for bandwidth with a newly selected one, while a
// fully loaded realization is not forced through a full reload if switched back to.
void cancelPendingTransfers();
// Fetches and stores exactly the given time steps of a dynamic property, in parallel, without pulling in
// the rest of the time series. See RifReaderSumoGridProperty::prefetchDynamicResult.
bool prefetchDynamicResult( const QString& resultName, const std::vector<size_t>& stepIndices ) override;
QString locationOnDisc() const override;
QString dataLoadingText() const override;
protected:
void defineUiOrdering( QString uiConfigName, caf::PdmUiOrdering& uiOrdering ) override;
private:
// Download the roff grid blob for this realization and parse it into the case data.
bool downloadAndParseGrid();
// Register the results and attach the property reader. Shared by the two ways the grid can end up on
// this case: parsed and kept, or parsed and replaced by the grid shared with the other realizations.
void finalizeCaseSetup();
// Discover the available grid properties from Sumo, register them as cell results and attach a reader
// that fetches the property data on demand. Static properties only in this first version.
void registerSumoGridProperties();
// Start the transfer of the property time step about to be displayed. Called before the grid download so
// the two run in parallel; only the decode needs the grid.
//
// The values go to the reader, which does not exist yet at this point. That is safe: arrivals come through
// the event loop, and runOnTransferThreadBlocking holds the grid download on a semaphore without
// dispatching events, so nothing lands before the case is set up.
void startPropertyFetch();
// The property and time step a view of this case is about to show. False when no view has picked one yet,
// leaving the on demand path to fetch whatever is asked for.
bool propertyToFetch( QString& propertyName, size_t& stepIndex, QString& isoDateOrInterval ) const;
// Callbacks handed to m_propertyReader, so the reader itself stays free of GUI code. Shows the loading
// banner and status bar message, and redraws the views once a time step has arrived.
void onPropertyPendingChanged( const QString& message );
void onPropertyTimeStepArrived( const QString& propertyName, size_t stepIndex, bool ok );
// Turns the reader's plain completed/total/description numbers into an actual progress dialog: opens one
// the first time total is non-zero (or replaces it with a freshly-sized one if total grew while a dialog
// was already open - caf::ProgressInfo's maximum cannot change after construction), ticks it as completed
// grows, and closes it once total is reported back as 0.
void onPropertyBatchProgressChanged( size_t completed, size_t total, const QString& description );
// Redraw the views of this case, so arrived values become visible.
void scheduleRedrawOfViews();
private:
caf::PdmPtrField<RimSumoDataSource*> m_sumoDataSource;
caf::PdmField<QString> m_sumoCaseId;
caf::PdmField<QString> m_ensembleName;
caf::PdmField<QString> m_gridName;
caf::PdmField<int> m_realization;
QPointer<RiaSumoConnector> m_sumoConnector;
// The same reader the cell results hold, set in registerSumoGridProperties. Time steps fetched before it
// existed are handed to it.
cvf::ref<RifReaderSumoGridProperty> m_propertyReader;
// The properties Sumo reports for this realization. Fetched before the grid and reused when they are
// registered, so the request is made once.
std::vector<SumoGridPropertyInfo> m_propertyInfos;
// The time step startPropertyFetch put in flight, until the reader exists and takes it over. Handed to
// the reader as pending so it is reported to the user and not requested a second time.
std::optional<std::pair<QString, size_t>> m_fetchInFlight;
// Identifies the transfer startPropertyFetch issues before the reader (and its own lifetime token) exists,
// so it can still be cancelled from closeReservoirCase. Recreated on every close, so a transfer left over
// from a previous open is never cancelled by a later one.
std::shared_ptr<bool> m_lifetimeToken;
// A redraw can read another time step and arrive back here through the reader's callback. Coalesce
// rather than recurse.
bool m_isRedrawingViews = false;
bool m_hasMissedViewRedraw = false;
// One single dialog for however many time steps the reader currently has pending, so waiting for a run
// of single steps in a row does not pop open and close a new dialog for each one. Owned here rather than
// in the reader itself, which reports only plain numbers via onPropertyBatchProgressChanged - no GUI code
// lives in RifReaderSumoGridProperty. Null when nothing is pending.
std::unique_ptr<caf::ProgressInfo> m_batchProgress;
size_t m_batchProgressShownTotal = 0; // What m_batchProgress was actually constructed with.
};
@@ -1,7 +1,6 @@
set(SOURCE_GROUP_SOURCE_FILES
${CMAKE_CURRENT_LIST_DIR}/RimSummaryEnsembleSumo.cpp
${CMAKE_CURRENT_LIST_DIR}/RimSummaryCaseSumo.cpp
${CMAKE_CURRENT_LIST_DIR}/RimSumoDataSource.cpp
)
list(APPEND CODE_SOURCE_FILES ${SOURCE_GROUP_SOURCE_FILES})
@@ -30,13 +30,13 @@
#include "RifEclipseSummaryAddress.h"
#include "Cloud/RimCloudDataSourceCollection.h"
#include "Cloud/RimSumoDataSource.h"
#include "RimProject.h"
#include "RimSummaryCaseMainCollection.h"
#include "RimSummaryCaseSumo.h"
#include "RimSummaryCurve.h"
#include "RimSummaryMultiPlot.h"
#include "RimSummaryPlot.h"
#include "RimSumoDataSource.h"
#include "RiuPlotCurve.h"
@@ -163,6 +163,20 @@ RimSummaryEnsembleSumo::RimSummaryEnsembleSumo()
m_lifetimeToken = std::make_shared<bool>( true );
}
//--------------------------------------------------------------------------------------------------
/// Aborts any transfers this ensemble still has in flight, see RiaSumoConnector::cancelGroup.
///
/// Rotate m_lifetimeToken before calling cancelGroup(), not after: see
/// RifReaderSumoGridProperty::~RifReaderSumoGridProperty() for why a reentrant, synchronous delivery
/// during cancelGroup() must observe an already-invalidated token, not the one about to be cancelled.
//--------------------------------------------------------------------------------------------------
RimSummaryEnsembleSumo::~RimSummaryEnsembleSumo()
{
void* lifetimeTokenKey = m_lifetimeToken.get();
m_lifetimeToken.reset();
if ( m_sumoConnector ) m_sumoConnector->cancelGroup( lifetimeTokenKey );
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
@@ -298,14 +312,16 @@ void RimSummaryEnsembleSumo::loadEnsembleParameters()
std::weak_ptr<bool> isAlive = m_lifetimeToken;
m_sumoConnector->summary().parameterDataAsync( sumoCaseId,
sumoEnsembleName,
[this, isAlive, parametersKey]( const QByteArray& contents )
{
if ( isAlive.expired() ) return;
m_sumoConnector->summary().parameterDataAsync(
sumoCaseId,
sumoEnsembleName,
[this, isAlive, parametersKey]( const QByteArray& contents )
{
if ( isAlive.expired() ) return;
onParameterDataReceived( parametersKey, contents );
} );
onParameterDataReceived( parametersKey, contents );
},
m_lifetimeToken.get() );
}
//--------------------------------------------------------------------------------------------------
@@ -370,18 +386,18 @@ void RimSummaryEnsembleSumo::prefetchSummaryData( const std::vector<RifEclipseSu
std::weak_ptr<bool> isAlive = m_lifetimeToken;
m_sumoConnector->summary().vectorDataAsync( sumoCaseId,
sumoEnsembleName,
vectorNamesToFetch,
[this, isAlive, sumoCaseId, sumoEnsembleName]( const QString& vectorName,
const QByteArray& contents )
{
// The request outlived the ensemble that asked for it.
if ( isAlive.expired() ) return;
m_sumoConnector->summary().vectorDataAsync(
sumoCaseId,
sumoEnsembleName,
vectorNamesToFetch,
[this, isAlive, sumoCaseId, sumoEnsembleName]( const QString& vectorName, const QByteArray& contents )
{
// The request outlived the ensemble that asked for it.
if ( isAlive.expired() ) return;
onVectorDataReceived( ParquetKey{ sumoCaseId, sumoEnsembleName, vectorName, false },
contents );
} );
onVectorDataReceived( ParquetKey{ sumoCaseId, sumoEnsembleName, vectorName, false }, contents );
},
m_lifetimeToken.get() );
}
//--------------------------------------------------------------------------------------------------
@@ -924,6 +940,13 @@ void RimSummaryEnsembleSumo::onLoadDataAndUpdate()
{
if ( m_sumoDataSource() )
{
// Rebuilding against realizations that have not been fetched would read "not known yet" as "nothing
// selected" and delete every realization case of this ensemble - and unlike the grid ensembles, these
// cases are written to the project file. See RimCloudDataSourceCollection::refreshDataSourcesFromSumo,
// which fetches them when a project is loaded, and RimSumoDataSource::updateGridCaseEnsembles, which
// guards the grid ensembles the same way.
const bool realizationsAreKnown = m_sumoDataSource->hasFetchedRealizations();
std::set<int> selectedRealizations;
for ( const auto& realizationId : m_sumoDataSource->selectedRealizationIds() )
{
@@ -942,8 +965,10 @@ void RimSummaryEnsembleSumo::onLoadDataAndUpdate()
}
// Update the realization cases whenever the selected set changes (added, removed or swapped),
// so editing the realizations on the data source updates the summary plot.
if ( selectedRealizations != currentRealizations )
// so editing the realizations on the data source updates the summary plot. Cases are only ever created
// when something is selected, which needs the realizations to be known, so this guard cannot suppress a
// rebuild that would have done anything but delete.
if ( realizationsAreKnown && selectedRealizations != currentRealizations )
{
// Update incrementally: a case whose realization is still selected is kept alive and reused.
// Widening the selection therefore deletes nothing, which keeps the curves referring to those
@@ -64,6 +64,9 @@ public:
public:
RimSummaryEnsembleSumo();
// Aborts any transfers this ensemble still has in flight, see RiaSumoConnector::cancelGroup.
~RimSummaryEnsembleSumo() override;
void setSumoDataSource( RimSumoDataSource* sumoDataSource );
void onRealizationSelectionChanged();
@@ -27,12 +27,12 @@
#include "RiaQStringFormatter.h"
#include "Summary/RiaSummaryTools.h"
#include "Cloud/RimSummaryCaseSumo.h"
#include "RifCaseRealizationParametersReader.h"
#include "RifEclipseSummaryTools.h"
#include "RifOpmCommonSummary.h"
#include "RifOpmSummaryTools.h"
#include "RifSummaryCaseRestartSelector.h"
#include "Sumo/RimSummaryCaseSumo.h"
#ifdef USE_HDF5
#include "RifHdf5SummaryExporter.h"
@@ -2,14 +2,25 @@
#include "RifRoffFileTools.h"
#include "RiaDefines.h"
#include "RiaTestDataDirectory.h"
#include "RigActiveCellInfo.h"
#include "RigEclipseCaseData.h"
#include "RigMainGrid.h"
#include "Reader.hpp"
#include "cvfBoundingBox.h"
#include "cvfObject.h"
#include <QDir>
#include <QFile>
#include <algorithm>
#include <cmath>
#include <fstream>
#include <vector>
TEST( RifRoffFileTools, ComputeZoneValuesFromSubgridsBasic )
{
@@ -88,3 +99,292 @@ TEST( RifRoffFileTools, ReadSubgridsFromRoffFile )
EXPECT_EQ( "Middle Zone", names[1] );
EXPECT_EQ( "Bottom Zone", names[2] );
}
//--------------------------------------------------------------------------------------------------
// computeActiveCellMatrixIndex renumbers the active cells in place: every non-zero entry is replaced
// by its running active index, every zero entry by -1. The return value is the number of active cells.
//--------------------------------------------------------------------------------------------------
TEST( RifRoffFileTools, ComputeActiveCellMatrixIndexMixed )
{
std::vector<int> activeCells = { 1, 0, 1, 1, 0 };
const size_t activeCount = RifRoffFileTools::computeActiveCellMatrixIndex( activeCells );
EXPECT_EQ( 3u, activeCount );
EXPECT_EQ( std::vector<int>( { 0, -1, 1, 2, -1 } ), activeCells );
}
TEST( RifRoffFileTools, ComputeActiveCellMatrixIndexAllActive )
{
std::vector<int> activeCells = { 1, 1, 1 };
EXPECT_EQ( 3u, RifRoffFileTools::computeActiveCellMatrixIndex( activeCells ) );
EXPECT_EQ( std::vector<int>( { 0, 1, 2 } ), activeCells );
}
TEST( RifRoffFileTools, ComputeActiveCellMatrixIndexAllInactive )
{
std::vector<int> activeCells = { 0, 0, 0 };
EXPECT_EQ( 0u, RifRoffFileTools::computeActiveCellMatrixIndex( activeCells ) );
EXPECT_EQ( std::vector<int>( { -1, -1, -1 } ), activeCells );
}
TEST( RifRoffFileTools, ComputeActiveCellMatrixIndexEmpty )
{
std::vector<int> activeCells;
EXPECT_EQ( 0u, RifRoffFileTools::computeActiveCellMatrixIndex( activeCells ) );
EXPECT_TRUE( activeCells.empty() );
}
TEST( RifRoffFileTools, ComputeActiveCellMatrixIndexTreatsAnyNonZeroAsActive )
{
// The roff active array is a byte flag, but the check is != 0, so any non-zero value counts.
std::vector<int> activeCells = { 5, 0, -3 };
EXPECT_EQ( 2u, RifRoffFileTools::computeActiveCellMatrixIndex( activeCells ) );
EXPECT_EQ( std::vector<int>( { 0, -1, 1 } ), activeCells );
}
//--------------------------------------------------------------------------------------------------
// hasGridData reports whether the file carries an actual grid, by looking for the cornerLines data.
//--------------------------------------------------------------------------------------------------
TEST( RifRoffFileTools, HasGridDataTrueForGridFile )
{
QDir baseFolder( TEST_DATA_DIR );
auto filePath = baseFolder.absoluteFilePath( "RifRoffReader/simple_grid.roff" );
ASSERT_TRUE( QFile::exists( filePath ) );
EXPECT_TRUE( RifRoffFileTools::hasGridData( filePath ) );
}
TEST( RifRoffFileTools, HasGridDataFalseForParameterOnlyFile )
{
QDir baseFolder( TEST_DATA_DIR );
auto filePath = baseFolder.absoluteFilePath( "RifRoffReader/facies_info.roff" );
ASSERT_TRUE( QFile::exists( filePath ) );
EXPECT_FALSE( RifRoffFileTools::hasGridData( filePath ) );
}
TEST( RifRoffFileTools, HasGridDataFalseForMissingFile )
{
QDir baseFolder( TEST_DATA_DIR );
auto filePath = baseFolder.absoluteFilePath( "RifRoffReader/this_file_does_not_exist.roff" );
ASSERT_FALSE( QFile::exists( filePath ) );
EXPECT_FALSE( RifRoffFileTools::hasGridData( filePath ) );
}
//--------------------------------------------------------------------------------------------------
// openGridFile on a stream is the entry point used when the roff data is not a file on disk, e.g. a
// blob downloaded from Sumo.
//--------------------------------------------------------------------------------------------------
TEST( RifRoffFileTools, OpenGridFileFromStream )
{
QDir baseFolder( TEST_DATA_DIR );
auto filePath = baseFolder.absoluteFilePath( "RifRoffReader/simple_grid.roff" );
ASSERT_TRUE( QFile::exists( filePath ) );
std::ifstream stream( filePath.toStdString(), std::ios::binary );
ASSERT_TRUE( stream.good() );
cvf::ref<RigEclipseCaseData> caseData = new RigEclipseCaseData( nullptr );
QString errorMessages;
ASSERT_TRUE( RifRoffFileTools::openGridFile( stream, caseData.p(), &errorMessages ) );
EXPECT_TRUE( errorMessages.isEmpty() );
auto* mainGrid = caseData->mainGrid();
ASSERT_TRUE( mainGrid != nullptr );
EXPECT_EQ( 2u, mainGrid->cellCountI() );
EXPECT_EQ( 2u, mainGrid->cellCountJ() );
EXPECT_EQ( 2u, mainGrid->cellCountK() );
EXPECT_EQ( 8u, mainGrid->cellCount() );
// The fixture marks two of the eight cells inactive.
auto* activeCellInfo = caseData->activeCellInfo( RiaDefines::PorosityModelType::MATRIX_MODEL );
ASSERT_TRUE( activeCellInfo != nullptr );
EXPECT_EQ( 6u, activeCellInfo->reservoirActiveCellCount() );
// The pillars are vertical and 100 apart, and the translate tag offsets x by 1000 and y by 2000.
// Depths are negated when the corners are built, so z runs from -100 up to 0. The extent is taken
// from the nodes rather than from RigMainGrid::boundingBox(), which is only filled in by
// computeCachedData() - openGridFile leaves that to its caller.
ASSERT_EQ( 8u * 8u, mainGrid->nodes().size() );
cvf::BoundingBox bb;
for ( const auto& node : mainGrid->nodes() )
{
bb.add( node );
}
EXPECT_NEAR( 1000.0, bb.min().x(), 1e-6 );
EXPECT_NEAR( 1200.0, bb.max().x(), 1e-6 );
EXPECT_NEAR( 2000.0, bb.min().y(), 1e-6 );
EXPECT_NEAR( 2200.0, bb.max().y(), 1e-6 );
EXPECT_NEAR( -100.0, bb.min().z(), 1e-6 );
EXPECT_NEAR( 0.0, bb.max().z(), 1e-6 );
}
TEST( RifRoffFileTools, OpenGridFileFromStreamFailsWithoutGridData )
{
// A property-only roff has no dimensions/translate/scale/cornerLines, so the import must fail
// cleanly and report why rather than leaving a half-built grid behind.
QDir baseFolder( TEST_DATA_DIR );
auto filePath = baseFolder.absoluteFilePath( "RifRoffReader/two_properties.roff" );
ASSERT_TRUE( QFile::exists( filePath ) );
std::ifstream stream( filePath.toStdString(), std::ios::binary );
ASSERT_TRUE( stream.good() );
cvf::ref<RigEclipseCaseData> caseData = new RigEclipseCaseData( nullptr );
QString errorMessages;
EXPECT_FALSE( RifRoffFileTools::openGridFile( stream, caseData.p(), &errorMessages ) );
EXPECT_FALSE( errorMessages.isEmpty() );
}
//--------------------------------------------------------------------------------------------------
// propertyValuesFromStream reads one property out of an in-memory roff blob. Among the arrays whose
// length matches the grid cell count it prefers the one whose keyword matches the requested name
// (case-insensitively), and it masks inactive cells with HUGE_VAL.
//--------------------------------------------------------------------------------------------------
namespace
{
// Open the 2x2x2 fixture grid, so a property blob has something to be read against.
cvf::ref<RigEclipseCaseData> loadFixtureGrid()
{
QDir baseFolder( TEST_DATA_DIR );
auto filePath = baseFolder.absoluteFilePath( "RifRoffReader/simple_grid.roff" );
if ( !QFile::exists( filePath ) ) return nullptr;
std::ifstream stream( filePath.toStdString(), std::ios::binary );
if ( !stream.good() ) return nullptr;
cvf::ref<RigEclipseCaseData> caseData = new RigEclipseCaseData( nullptr );
if ( !RifRoffFileTools::openGridFile( stream, caseData.p(), nullptr ) ) return nullptr;
return caseData;
}
// The values of the cells the fixture marks active, with the inactive ones dropped. Both the property
// array and the active array are reordered the same way, so the pairing survives and this can be
// compared without depending on the roff-to-reservoir index mapping.
std::vector<double> finiteValuesSorted( const std::vector<double>& values )
{
std::vector<double> finite;
for ( double v : values )
{
if ( v != HUGE_VAL ) finite.push_back( v );
}
std::sort( finite.begin(), finite.end() );
return finite;
}
std::vector<double> readProperty( const QString& propertyName, RigEclipseCaseData* caseData, bool* ok )
{
QDir baseFolder( TEST_DATA_DIR );
auto filePath = baseFolder.absoluteFilePath( "RifRoffReader/two_properties.roff" );
std::ifstream stream( filePath.toStdString(), std::ios::binary );
std::vector<double> values;
*ok = RifRoffFileTools::propertyValuesFromStream( stream, caseData, propertyName, &values );
return values;
}
} // namespace
TEST( RifRoffFileTools, PropertyValuesFromStreamSelectsRequestedProperty )
{
cvf::ref<RigEclipseCaseData> caseData = loadFixtureGrid();
ASSERT_TRUE( caseData.notNull() );
bool ok = false;
auto values = readProperty( "SOIL", caseData.p(), &ok );
ASSERT_TRUE( ok );
ASSERT_EQ( 8u, values.size() );
// The two inactive cells are masked, the remaining six carry the SOIL values, not the SWAT ones.
EXPECT_EQ( std::vector<double>( { 110.0, 120.0, 130.0, 140.0, 150.0, 160.0 } ), finiteValuesSorted( values ) );
EXPECT_EQ( 2, std::count( values.begin(), values.end(), HUGE_VAL ) );
}
TEST( RifRoffFileTools, PropertyValuesFromStreamSelectsTheOtherProperty )
{
cvf::ref<RigEclipseCaseData> caseData = loadFixtureGrid();
ASSERT_TRUE( caseData.notNull() );
bool ok = false;
auto values = readProperty( "SWAT", caseData.p(), &ok );
ASSERT_TRUE( ok );
EXPECT_EQ( std::vector<double>( { 10.0, 20.0, 30.0, 40.0, 50.0, 60.0 } ), finiteValuesSorted( values ) );
}
TEST( RifRoffFileTools, PropertyValuesFromStreamMatchesNameCaseInsensitively )
{
cvf::ref<RigEclipseCaseData> caseData = loadFixtureGrid();
ASSERT_TRUE( caseData.notNull() );
bool ok = false;
auto values = readProperty( "soil", caseData.p(), &ok );
ASSERT_TRUE( ok );
EXPECT_EQ( std::vector<double>( { 110.0, 120.0, 130.0, 140.0, 150.0, 160.0 } ), finiteValuesSorted( values ) );
}
TEST( RifRoffFileTools, PropertyValuesFromStreamFallsBackToFirstCandidate )
{
// No array matches the requested name, so the first array of the right length is used. This is the
// documented fallback, and it keeps a blob whose keyword does not match the result name readable.
cvf::ref<RigEclipseCaseData> caseData = loadFixtureGrid();
ASSERT_TRUE( caseData.notNull() );
bool ok = false;
auto values = readProperty( "NO_SUCH_PROPERTY", caseData.p(), &ok );
ASSERT_TRUE( ok );
EXPECT_EQ( std::vector<double>( { 10.0, 20.0, 30.0, 40.0, 50.0, 60.0 } ), finiteValuesSorted( values ) );
}
TEST( RifRoffFileTools, PropertyValuesFromStreamRejectsMissingArguments )
{
QDir baseFolder( TEST_DATA_DIR );
auto filePath = baseFolder.absoluteFilePath( "RifRoffReader/two_properties.roff" );
cvf::ref<RigEclipseCaseData> caseData = loadFixtureGrid();
ASSERT_TRUE( caseData.notNull() );
std::vector<double> values;
{
std::ifstream stream( filePath.toStdString(), std::ios::binary );
EXPECT_FALSE( RifRoffFileTools::propertyValuesFromStream( stream, nullptr, "SOIL", &values ) );
}
{
std::ifstream stream( filePath.toStdString(), std::ios::binary );
EXPECT_FALSE( RifRoffFileTools::propertyValuesFromStream( stream, caseData.p(), "SOIL", nullptr ) );
}
}
TEST( RifRoffFileTools, PropertyValuesFromStreamRejectsMismatchedCellCount )
{
// facies_info.roff declares a 46x112x242 grid and carries only 4 values, so nothing matches the
// cell count of the 2x2x2 fixture grid.
cvf::ref<RigEclipseCaseData> caseData = loadFixtureGrid();
ASSERT_TRUE( caseData.notNull() );
QDir baseFolder( TEST_DATA_DIR );
auto filePath = baseFolder.absoluteFilePath( "RifRoffReader/facies_info.roff" );
std::ifstream stream( filePath.toStdString(), std::ios::binary );
std::vector<double> values;
EXPECT_FALSE( RifRoffFileTools::propertyValuesFromStream( stream, caseData.p(), "composite", &values ) );
}
@@ -0,0 +1,56 @@
roff-asc
#ROFF file with a minimal 2x2x2 grid, hand written for unit tests#
tag filedata
int byteswaptest 1
char filetype "grid"
char creationDate "01/01/2026 00:00:00"
endtag
tag version
int major 2
int minor 0
endtag
tag dimensions
int nX 2
int nY 2
int nZ 2
endtag
tag translate
float xoffset 1000.0
float yoffset 2000.0
float zoffset 0.0
endtag
tag scale
float xscale 1.0
float yscale 1.0
float zscale 1.0
endtag
tag cornerLines
array float data 54
0 0 0 0 0 1000
0 100 0 0 100 1000
0 200 0 0 200 1000
100 0 0 100 0 1000
100 100 0 100 100 1000
100 200 0 100 200 1000
200 0 0 200 0 1000
200 100 0 200 100 1000
200 200 0 200 200 1000
endtag
tag zvalues
array byte splitEnz 27
1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1
array float data 27
0 50 100 0 50 100
0 50 100 0 50 100
0 50 100 0 50 100
0 50 100 0 50 100
0 50 100
endtag
tag active
array byte data 8
1 1 1 1 1 1 0 0
endtag
tag eof
endtag
@@ -0,0 +1,30 @@
roff-asc
#ROFF file with two cell properties, hand written for unit tests#
tag filedata
int byteswaptest 1
char filetype "parameter"
char creationDate "01/01/2026 00:00:00"
endtag
tag version
int major 2
int minor 0
endtag
tag dimensions
int nX 2
int nY 2
int nZ 2
endtag
tag parameter
char name "SWAT"
array float data 8
10 20 30 40 50 60
70 80
endtag
tag parameter
char name "SOIL"
array float data 8
110 120 130 140 150 160
170 180
endtag
tag eof
endtag
@@ -157,6 +157,13 @@ RiuViewer::RiuViewer( QWidget* parent )
m_filterLabel->setFont( font );
m_showFilterLabel = false;
m_loadingLabel = new QLabel();
m_loadingLabel->setFrameShape( QFrame::NoFrame );
m_loadingLabel->setAlignment( Qt::AlignCenter );
m_loadingLabel->setObjectName( "LoadingLabel" );
m_loadingLabel->setFont( font );
m_showLoadingLabel = false;
// Animation progress bar
m_animationProgress = new caf::QStyledProgressBar( "AnimationProgress" );
m_animationProgress->setFormat( "Time Step: %v/%m" );
@@ -193,6 +200,7 @@ RiuViewer::RiuViewer( QWidget* parent )
m_histogramWidget->setFont( regTestFont );
m_zScaleLabel->setFont( regTestFont );
m_filterLabel->setFont( regTestFont );
m_loadingLabel->setFont( regTestFont );
}
// When a context menu is created in the viewer is, and the action triggered is displaying a dialog,
@@ -566,6 +574,14 @@ void RiuViewer::paintOverlayItems( QPainter* painter )
m_filterLabel->render( painter, pos );
}
if ( m_showLoadingLabel ) // Loading banner, centered across the top where it cannot be missed
{
QSize size( m_loadingLabel->sizeHint().width(), m_loadingLabel->sizeHint().height() );
QPoint pos( ( trueWidth - size.width() ) / 2, margin + edgeAxisFrameBorderHeight );
m_loadingLabel->resize( size.width(), size.height() );
m_loadingLabel->render( painter, pos );
}
if ( !m_cursorPositionDomainCoords.isUndefined() )
{
if ( mainCamera() )
@@ -654,6 +670,31 @@ void RiuViewer::setFilterText( const QString& text )
m_filterLabel->setText( text );
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RiuViewer::showLoadingLabel( bool enable )
{
if ( m_showLoadingLabel == enable ) return;
m_showLoadingLabel = enable;
// Nothing else is redrawing while data is on its way, so ask for it here.
update();
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
void RiuViewer::setLoadingText( const QString& text )
{
if ( m_loadingLabel->text() == text ) return;
m_loadingLabel->setText( text );
if ( m_showLoadingLabel ) update();
}
//--------------------------------------------------------------------------------------------------
///
//--------------------------------------------------------------------------------------------------
@@ -1524,6 +1565,8 @@ void RiuViewer::updateOverlayItemsStyle()
caf::StyleSheetTools::createFrameStyleSheet( "QLabel", "ZScaleLabel", contrastColor, backgroundColor, backgroundColor ) );
m_filterLabel->setStyleSheet(
caf::StyleSheetTools::createFrameStyleSheet( "QLabel", "FilterLabel", contrastColor, backgroundColor, backgroundColor ) );
m_loadingLabel->setStyleSheet(
caf::StyleSheetTools::createFrameStyleSheet( "QLabel", "LoadingLabel", contrastColor, backgroundColor, backgroundColor ) );
m_histogramWidget->setStyleSheet(
caf::StyleSheetTools::createFrameStyleSheet( "QWidget", "HistogramWidget", contrastColor, backgroundColor, backgroundFrameColor ) );
@@ -92,6 +92,11 @@ public:
void showFilterLabel( bool enable );
void setFilterText( const QString& text );
// A banner across the top of the view, for telling the user data is on its way. Deliberately not the
// info text, which is small, in a corner, and can be switched off.
void showLoadingLabel( bool enable );
void setLoadingText( const QString& text );
void showHistogram( bool enable );
void setHistogram( double min, double max, const std::vector<size_t>& histogram );
void setHistogramPercentiles( double pmin, double pmax, double mean );
@@ -187,6 +192,9 @@ private:
QLabel* m_filterLabel;
bool m_showFilterLabel;
QLabel* m_loadingLabel;
bool m_showLoadingLabel;
caf::QStyledProgressBar* m_animationProgress;
caf::QStyledProgressBar* m_animationProgressCompView;
bool m_showAnimProgress;
+141
View File
@@ -0,0 +1,141 @@
# Cloud Data Architecture
How ResInsight retrieves ensemble grid and summary data from SUMO. Three tiers are involved:
SUMO itself (Equinor's cloud store for FMU results), the local `ri-cloud-api` service that
ResInsight launches as its own **`ri-cloud-api` process** (a separate OS process from ResInsight
itself, running a uvicorn FastAPI service), and the C++ code in ResInsight that talks to it and
turns the results into grid cases and summary ensembles.
See also [cloud-service-api.md](cloud-service-api.md) for how the Python service is installed and
launched.
## Overview
```mermaid
architecture-beta
group local(server)[ResInsight Process]
group localServer(server)[ri cloud api Process]
group sumo(cloud)[SUMO python packages]
group azure(cloud)[Azure]
service connector(server)[RiaSumoConnector] in local
service riCore(disk)[ResInsight Case and Summary Data] in local
service cloudApi(server)[uvicorn FastAPI service] in localServer
service sumoPythonWrapper(internet)[sumo python wrapper Server Auth and SAS] in sumo
service fmuSumoExplorer(database)[SUMO Explorer Search Indexed FMU Metadata] in sumo
service blobStore(disk)[Blob Storage] in azure
junction gaplocalServer
junction gapSumoPythonWrapper
junction gapFmuSumoExplorer
junction gapBlobStore
connector:R -- L:gaplocalServer
gaplocalServer:R -- L:cloudApi
cloudApi:T -- L:gapSumoPythonWrapper
gapSumoPythonWrapper:R -- L:sumoPythonWrapper
cloudApi:R -- L:gapFmuSumoExplorer
gapFmuSumoExplorer:R -- L:fmuSumoExplorer
connector:T -- B:blobStore
connector:B --> T:riCore
```
## Components
### SUMO (Equinor cloud store)
- Stores FMU (Fast Model Update) ensemble results: case/ensemble/realization metadata, grid
geometry and grid property blobs (roff), summary vectors and ensemble parameters (parquet).
- **Sumo core** is the backend proper: it owns the ElasticSearch index (case/ensemble/realization/
object metadata and search) and exposes the API in front of it — including authentication and
issuing the short-lived SAS (Shared Access Signature) tokens used to download blobs. It is
written in Node.js and is self-hosted on Azure; ElasticSearch is not a separate system reached
independently, it lives behind this one API.
- **`sumo-wrapper-python`** is a thin Python wrapper around that API: it makes the Sumo API easier
to call from Python and provides the basic plumbing — authentication and retry handling — without
adding any FMU domain knowledge of its own.
- **`fmu-sumo`'s `Explorer`** is built on top of `sumo-wrapper-python`. It is an FMU-aware Python
library for reading FMU results out of Sumo, offering an intuitive, domain-level interface
(assets, cases, ensembles, realizations, grid/summary objects) instead of raw API/ElasticSearch
calls, and is what `ri-cloud-api` actually calls to look things up.
- The blob bytes themselves are **not** served by the Sumo core API in the current solution — a
lookup returns a SAS token and a blob-store base URI, and `RiaSumoConnector` downloads the blob
directly from **Azure Blob Storage**. `fmu-sumo`'s `Explorer` objects *can* also hand back blob
bytes/dataframes directly (e.g. `.blob`, `.values` on a search result), which would let
`ri-cloud-api` serve the bytes itself instead of a SAS token — that path is not used today, but
is a possible future option if there is a reason to route the data through the service rather
than fetching it directly from Azure.
- Authentication to SUMO is Azure AD (Entra ID) OAuth2 Authorization Code Flow, handled in
ResInsight by `RiaCloudConnector`/`RiaSumoConnector` (opens a browser once for sign-in, then
reuses the token).
### `ri-cloud-api` (local Python service, submodule `scripts/ri-cloud-api`)
- A FastAPI app (`ri_cloud_api.main:app`) that ResInsight starts on demand as a local subprocess
(`RiaCloudApiService`, via `uvicorn` on `127.0.0.1:<free port>`) once the user signs in.
- Bridges ResInsight's simple REST calls to the SUMO client libraries:
- `fmu-sumo` — the Explorer, an FMU-aware library for reading FMU results/metadata from Sumo
(case/ensemble/realization/object search), built on top of `sumo-wrapper-python`.
- `sumo-wrapper-python` — the thin Python wrapper for the Sumo API that `fmu-sumo` (and
`ri-cloud-api`, where needed directly) uses underneath: authentication, retries, and
SAS-token/blob-access-info retrieval.
- `ri-cloud-core-utils` and `ri-cloud-services` — the two workspace libraries under
`libs/core_utils` and `libs/services` that implement the service logic on top of those.
- Routers exposed (`ri_cloud_api/primary/routers`): `explore` (assets/cases/ensembles/
realizations), `grids` (grid + grid property blob ids), `timeseries` (summary vector blob ids,
aggregated on demand), `parameters` (ensemble parameters), `surfaces`, `polygons`,
`blob_access` (SAS token + blob-store base URI lookup), `health` (`/alive`).
- Runs entirely on `127.0.0.1`; nothing outside the local machine can reach it.
### ResInsight (C++ client side)
- `RiaCloudApiService` — starts/stops the local `uvicorn` process, finds a free port, and polls
`/alive` until the service is ready.
- `RiaSumoConnector` — owns the HTTP transport (OAuth2 token, a dedicated transfer thread,
blocking and async request helpers) and three data-specific delegates:
- `RiaSumoExplore` — assets, cases, ensembles, realizations.
- `RiaSumoGrid` — grid names/dimensions, grid geometry (roff), grid property blobs, sync and
async batched property time steps.
- `RiaSumoSummary` — summary vector names/values, ensemble parameters, sync and async.
- Blob retrieval is always two round trips, done through the connector:
1. `GET .../blobs/{id}/sas_token_and_blob_base_uri` on the local `ri-cloud-api` service →
`{ sasToken, blobStoreBaseUri }`.
2. `GET {blobStoreBaseUri}/{id}?{sasToken}` **directly against Azure Blob Storage** → the blob
bytes (roff grid/property data, or parquet summary/parameter data).
- Consumers:
- `RimRoffCaseSumo` + `RifReaderSumoGridProperty` — one Sumo grid realization becomes a
`RimEclipseCase`. The grid geometry is downloaded and parsed up front; grid properties are
fetched lazily/asynchronously the first time a time step is displayed, decoded straight into
`RigCaseCellResultsData` (no long-lived copy of the downloaded bytes is kept).
- `RimSummaryEnsembleSumo` — a Sumo ensemble becomes a `RimSummaryEnsemble`. Vector data and
ensemble parameters are fetched asynchronously (parquet, one blob per vector/parameter set)
and decoded into the summary case's readers.
## `RimRoffCaseSumo` (grid case)
```mermaid
flowchart LR
Ensemble[RimReservoirGridEnsembleSumo] --> Case[RimRoffCaseSumo]
Case -- "requests grid blob (sync)" --> Connector[RiaSumoConnector]
Connector -- "roff blobs" --> Case
Case -- "parses roff blob, creates" --> CaseData[RigEclipseCaseData]
Case -- attaches reader --> Reader[RifReaderSumoGridProperty]
Reader -- "fetches property time steps (async)" --> Connector
Reader -- registered as reader for --> Results[RigCaseCellResultsData]
CaseData --> Results
Connector -- "downloads blob (SAS token)" --> Sumo[(ri-cloud-api / Azure Blob Storage)]
```
## `RimSummaryEnsembleSumo` (summary ensemble)
```mermaid
flowchart LR
Ensemble[RimSummaryEnsembleSumo] -- "requests vectors / parameters (async)" --> Connector[RiaSumoConnector]
Connector -- "downloads blob (SAS token)" --> Sumo[(ri-cloud-api / Azure Blob Storage)]
Connector -- parquet blobs --> Ensemble
Ensemble -- "decodes via Arrow, distributes" --> Realizations[Per-realization summary readers]
Realizations --> Plots[Summary Plots]
```