Requests are queued by the completion queue thread and processed from the main
thread. A long running request, or an open progress dialog in the GUI, therefore
left every request unanswered for as long as that operation lasted, including
the liveness checks a client uses to decide whether ResInsight is still there.
Let a callback declare that it touches no application state, and serve those
directly on the completion queue thread instead of queueing them. GetVersion is
the only such callback, since it reads compile-time constants only. Everything
that touches the project or the data model is still processed on the main
thread, so no request can observe or modify the data model while another one is
running.
The allocated callback list is now used from both threads and is guarded by a
mutex. Shutdown takes a lock before flagging that it is in progress, so that it
waits for any dispatch already running on the completion queue thread and no new
request handler can be registered on a queue that is about to be shut down.
During a blocking call of 3.25 s, 65 pings were answered with a maximum latency
of 6.2 ms. Before this change none of them were answered until the call
completed.
A schedule generated from a full production history, or a large raw text block
imported into the event timeline, easily exceeds the gRPC default limit of 4 MB.
Such a call failed with "Received message larger than max (5119402 vs 4194304)"
even though both sides were healthy.
Raise the limit to 128 MB on the server and use a matching limit for the client
channels. A schedule of 19.5 MB is now returned without error.
Deleting PDM objects on a worker thread is not safe. PdmObjectHandle::prepareForDelete()
mutates state owned by other objects, it nulls the guarded pointers held by other objects
and clears m_pointersReferencingMe, an unsynchronised std::set that every PdmPointer
construction and destruction touches. Destroying an object off the main thread therefore
races with the main thread on the shared object graph. See issue 14491.
Profiling a summary ensemble teardown shows the mechanism does not pay for itself. Releasing
397 Drogon realizations takes 0.24 s sequentially and 0.24 s in parallel, and for a heavy
case the parallel release is slower than the sequential one, because free() is serialised
inside the allocator. The PDM bookkeeping itself is 0.4 to 2.8 percent of the teardown.
Remove the class and deleteChildrenAsync(), and delete synchronously instead. The call sites
that used clearWithoutDelete() and a manual delete loop to work around the race can now call
deleteChildren() directly. Add caf::PdmObjectHandleTools::deleteObjects() for the case where
the objects are no longer owned by a child array field.
The observer disconnection from issue 12262 does not depend on clearWithoutDelete(). ~Signal()
unregisters itself from every observer, so a deleted child detaches itself. That fix addressed
the async race, where ~Signal() mutated the observer list from a worker thread. The unit test
is updated to assert the observed signal count directly instead of relying on a crash.
Changed logging from info to debug for config imports, CAF logging, socket server, and gRPC server startup. OpenTelemetry initialization now only runs if enabled, with results logged at debug level.
* Python: avoid assigning same port number to multiple grpc sessions
* Add retry count when checking for port number file
* Use grpc to find and use port number
* Add test used to start several instances of resinsight at the same time
Testing up to 50 instances works well
* Python: allow launch_port == 0 to assign port by GRPC in Instance.launch().
Also allow longer wait before failing the port number file reading:
it can take some time to launch when launching lots of instances at
the same time.
Kristian Bendiksen
Magne Sjaastad
Gaute Lindkvist