Meltano Version

4.0.9

Python Version

NA

Bug scope

Configuration (settings parsing, validation, etc.)

Operating System

NA

Description

The dotenv_values function from the python-dotenv package can be set to None[^1]:

Variable without a value

A variable can have no value:

FOO

It results in dotenv_values associating that variable name with the value
None (e.g. {"FOO": None}. load_dotenv, on the other hand, simply ignores
such variables.

This shouldn't be confused with FOO=, in which case the variable is associated
with the empty string.

This means that if we have the following meltano.yml and .env

plugins:
  extractors:
  - name: tap-pypistats
    variant: edgarrmondragon
    pip_url: tap-pypistats
    config:
      packages:
      - tap-$TAP_NAME

TAP_NAME

produce this behavior during env var expansion

$ meltano --log-level warning config print tap-pypistats
{
  "packages": [
    "tap-None"
  ]
}

Decision

• Should we treat None as the empty string? i.e. ENV_VAR is the same as ENV_VAR=. Update: this was done in fix: Drop None values coming from .env #9825

• More interestingly: should ENV_VAR and its null value override lower-precedence setting stores, e.g.

  having TAP_MY_SETTING in .env and my_setting: val in meltano.yml will result in my_setting being set to null, i.e. unset (Bug: Null config values don't get passed to plugins, preventing Mappers from working as expected #6382, Allow default values to be overridden with null #2510).

Related

• Allow default values to be overridden with null #2510
• Bug: Null config values don't get passed to plugins, preventing Mappers from working as expected #6382
• fix: Drop None values coming from .env #9825

Code

plugins:
  extractors:
  - name: tap-pypistats
    variant: edgarrmondragon
    pip_url: tap-pypistats
    config:
      packages:
      - tap-$TAP_NAME

Link to Slack/Linen

No response