dbengine: the lifecycle becomes explicit - #23945

…gument

Three -W modes (pgd-tests, pgctest, mrgtest) and the registry test inside
-W unittest create a page cache or the page allocators and nothing else, but
read the engine's process-wide configuration to do so. They got it through the
embedder calling dbengine_init() first, an ordering rule between two public
calls for tests that never bring the engine up. Now dbengine_page_test(),
dbengine_cache_unittest() and dbengine_metrics_registry_unittest() take a
const struct dbengine_config * and set it themselves through a private setter
that resolves the 0-means-default fields exactly as dbengine_init() does (a
bare copy would leave cpus, the worker-thread count and pages_per_extent at 0,
which pgc_create() and pgd_init_arals() read). The daemon's side of the split:
netdata_conf_dbengine_resolved() snapshots the settings the daemon resolves
outside netdata.conf into its configuration and returns it, and
netdata_conf_dbengine_apply() becomes dbengine_init() of that. The four call
sites in main.c pass the resolved configuration instead of calling apply first;
the one inside -W unittest runs after test_dbengine() shut the engine down and
is served by the setter alone, which does not care.

The configuration flag is renamed dbengine_cfg_configured, and the accessor
dbengine_configured() replaces dbengine_initialized(): it means "the engine has
its configuration", not "the engine is running" (the lifecycle flags say that),
and a later change gives the two meanings different readers. Behaviour is
unchanged.

…opens a tier

The engine's process level (event loop, caches, metrics registry, thread) came
up as a side effect of the first dbengine_tier_init(), while dbengine_init()
only stored the configuration. Two rules existed only because of that: init had
to be called exactly once and refused a different configuration, and a tier
init had to know it might be the one to spawn. Now dbengine_init() copies the
configuration and brings the engine up, returning 0, the libuv error that
stopped the loop from coming up (the engine stays down), UV_EALREADY when it is
already up (the configuration is left alone) or UV_EIO after a shutdown. The
embedder decides what to do with a failure, the way it already does for a tier.
The spawn is a static helper of init; its unused tier parameter goes, and so do
the init-once check, the configuration comparison and the flag that answered
"was init called", which no longer has a reader. dbengine_tier_init() only
opens the static tier: its first act, before any write to that tier, is to read
the engine's lifecycle (fatal before init, UV_EIO after shutdown), so a refused
init leaves the static tier as it found it. dbengine_shutdown() is unchanged and
is now documented as init's counterpart, after every tier exit.

The daemon calls dbengine_init() through netdata_conf_dbengine_apply() (fatal
on any failure) right before its tier loop instead of before it reads
"storage tiers": the engine's registry preload counts the configured tiers,
which until now was read after init because the preload ran inside the first
tier init. One error path changes with that: a startup where every tier
directory cannot be created now brings the engine up and then dies with the
same "failed to initialize databases" fatal as before. rrd_init() no longer
calls apply for the unit tests; the three test drivers that use the engine
call it themselves, after the cache-floor check that needs the caches absent.

…guration back

With the caller-allocated tier gone, every dbengine_tier_init() lands on the
static tier for its number, and nothing refused a second init of a tier that
was already up: it rewrote the tier's path and quota, reset its transaction
counter and ran init_rrd_files() over datafiles the tier already owned. No
caller in the tree did that, but the contract did not say so either. Now a
tier that is up is refused with UV_EALREADY before anything is written to it,
the same way an engine that is up refuses a second dbengine_init(), and the
header says "once". The unit test asks for the refusal on the tier it runs on,
with a path that does not exist, so a re-opened tier would show up as errors
in every test that follows; it asks again after the shutdown, where UV_EIO is
the answer.

dbengine_init() copies the configuration before it spawns, because the caches
and the allocators read it as they come up; a spawn that fails does so before
they exist, so the previous configuration is put back and the engine is as
the header promises, as if never called. The rest is prose the change had left
behind: dbengine_shutdown() on an engine that never came up still closes it;
the configuration is read-only while the engine is up (the tests that need
only the configuration set it on a stopped engine); the private setter is not
guarded and says so; the defaults comment names all four resolved fields; the
README points from the tier lifecycle to the way up; the two -W driver comments
in main.c no longer say rrd_init() hands the engine its configuration; the
daemon's apply says a second call is fatal.

…complete

When uv_timer_init() failed inside the spawn, the unwind closed the async
handle (and the first timer) and then asserted that uv_loop_close() succeeds.
It cannot: a closed handle stays in the loop until a loop iteration runs its
close callback, so uv_loop_close() reports the loop busy and the assertion
aborted the process where dbengine_init() should have returned the error. The
unwind now runs the loop until the closes complete, then closes it, and on
Windows drops the async-ready flag it had raised. Reachable only when libuv
cannot allocate a handle; the same unwind existed before the spawn moved into
dbengine_init(), and the comment above the spawn now describes what it does.

The header promised more than the code delivers. "Once" read as a guarantee,
but the refusal only covers a tier that is up: dbengine_tier_exit() clears the
flag it reads, so a tier that came up and exited is not refused, and bringing
it up again is not safe either, because only dbengine_destroy() finalizes its
datafiles. "A refused init leaves the tier untouched" is true of the three
refusals and not of an init that fails opening the datafiles, which returns
UV_EIO with the configuration already written. And dbengine_shutdown() on an
engine that never came up does not close anything; it records the stop, so a
later dbengine_init() is refused. The header now says all three as they are,
and that two inits of the same tier must not overlap, which nothing serialises.

Two small things from the same review: the unwind of a failed spawn now drops
the Windows async-ready flag with an atomic release store before it closes the
handles, the order the event loop's own exit uses, and owns the closes itself;
and the unit test's refusal probe loses a parameter it never read.

…l; the post-shutdown probe runs on a tier that never came up

dbengine_tier_init() validated the configuration after the shut-down check
and before the two tier-state checks, so an invalid configuration aimed at a
tier that is up was fatal while the same configuration on a shut-down engine
was refused with an error. Validation now comes first: a bad configuration
is a programming error whatever the engine's state, and every refusal after
it leaves the tier untouched, as the header says.

The unit test's post-shutdown probe ran on tier 0, which by then had come up
and exited, so the "came up and exited" refusal answered the same UV_EIO the
shutdown refusal would have; a missing shutdown refusal would have passed
unnoticed. The probe now runs on tier 1, which never came up in the test
process, in a real empty directory, and also checks that no datafile appeared
there: only the shutdown refusal can answer, and a tier opened by mistake
would leave a datafile behind. The came_up comment says the bit is stored just
before active, not together with it.