Skip to content

SOPS displacing comments and causing YAML-breaking invalid syntax error and data loss #1068

@lucabello

Description

@lucabello

Hello everyone; I found a bug in the SOPS encryption, which you can replicate yourself.

Unencrypted file test.yaml:

this:
  is:
    an: example
  # comment

I use age encryption, so I set up the SOPS_AGE_RECIPIENTS and SOPS_AGE_KEY_FILE variables, and then run sops --encrypt --encrypted-regex an test.yaml, obtaining this:

this:
    is:
        an: ENC[AES256_GCM,data:7ew99FdoYQ==,iv:62k8jSLjg2l7YyorqOf9bobpgT7m+1qeEK66JpyObTE=,tag:caRiamKAO9TqZHj2qfKAgA==,type:str]
sops:
# comment
    kms: []
    gcp_kms: []
    azure_kv: []
    hc_vault: []
    age:
        - recipient: age107p580kfl3mlr2x7jqfhattzv0a3kd88ugjq2p5tsjz59jzdmdfsj2qwm2
          enc: |
            -----BEGIN AGE ENCRYPTED FILE-----
            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBZa3VuTFBucnV4QldrZW0z
            am9jTkw1ejZIUjBUVHpRTjRONUNTYnRXWFVRCncwdk5adnpDZktnK3JwVWU0R1o4
            UGlIOHpSdUpTQjBhWnMwcXAyZlhmTFEKLS0tIFk2bitXOHN2T2VIbHZzVnVlTVpn
            VGNUSkdFWENLSEFQdGh4OW1TMk1HZHMKvMCQo5SXkMk9HQ4Bv7n1UFj+IZDi0z8K
            1LJKCovXp7d+zceCwbVM9zfbleF7AvFuyWSbw7o11WpE4vtaD8fDWg==
            -----END AGE ENCRYPTED FILE-----
    lastmodified: "2022-05-31T14:38:08Z"
    mac: ENC[AES256_GCM,data:BNPYx43KNPdggGC/+qgHQfoMj+ctnrwjOpt+pOzoWcQCcS3IP1h0FtD/UcSc8mLTxS2+6gBBiFU8JIdnr8XwueIeGbcx0zEk8LZdspv0DuVQoC5AbiU4qnvCFRY7MIdmSKacG7zXVN4nQ2kW/Ip7jqhBAqZJl5cbUBcaQOgfGCo=,iv:aLQ+lhmgxzyrQphjknd1O1KHC6BVG1E0hM9tCfFs+J4=,tag:sHhaEqLU+/VRDRjTajAnwQ==,type:str]
    pgp: []
    encrypted_regex: an
    version: 3.7.3

As you can see, the # comment line is moved inside the sops metadata; whenever that file is decrypted, the comment will be gone (hence the data loss I mention in the title).

The reason why this is important is because my team is using some comment "tags" to mark code blocks, and the last tag disappears after encrypting-and-decrypting a file.

I did some extra testing to try to understand when this was happening, and I gathered some conclusions: it only happens if the indentation is at least the one in my example file (so at least 3) and the if the comment is "less indented" than the last line.

Sorry if this is a known issue and I missed it! :)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions