Skip to content

Add all-user domain and database listing commands - #5657

Open
thomas-berrio wants to merge 3 commits into
hestiacp:mainfrom
thomas-berrio:feature/4854-list-all-user-objects
Open

thomas-berrio wants to merge 3 commits into
hestiacp:mainfrom
thomas-berrio:feature/4854-list-all-user-objects

Conversation

@thomas-berrio

Copy link
Copy Markdown

Summary

Add server-wide listing commands intended for CLI and HTTP API consumers managing large HestiaCP installations:

  • v-list-all-web-domains [FORMAT]
  • v-list-all-mail-domains [FORMAT]
  • v-list-all-dns-domains [FORMAT]
  • v-list-all-databases [FORMAT]

Each command supports the existing plain, csv, shell, and json formats and adds the owning USER to every returned object or row.

Motivation

The current list commands are scoped to a single user. API clients that need a server-wide inventory must first list the users and then perform one request per user and object type.

These commands provide that inventory through one CLI or HTTP API request, which is especially useful when managing a large number of users and sites.

Implementation

  • Reuses v-list-users list and the existing user-scoped v-list-* commands through $BIN.
  • Keeps the existing commands and their output columns unchanged.
  • Shares aggregation logic in the dedicated func/list.sh helper.
  • Buffers the complete result before writing to stdout, so a child-command failure cannot return a valid-looking partial inventory.
  • Adds an admin-only list-all-user-objects API permission covering the four commands.
  • Installs the permission on fresh installations and adds it during the current upgrade path.
  • Documents all four commands.

This intentionally performs one existing list command per user. It avoids duplicating the parsing of Hestia configuration files and keeps the new output aligned with the existing commands.

Mail, DNS, and database listings are included because they use the same aggregation and authorization model as web domains, with no changes to their existing commands.

Tests

Added coverage for:

  • all four output formats;
  • web, mail, DNS, and database ownership;
  • multiple users in JSON output;
  • child-command error propagation with empty stdout;
  • the admin-only API permission;
  • rejection for non-admin users;
  • an HTTP API request using a restricted access key.

Local syntax, diff, error-propagation, and security mock checks pass. The complete HestiaCP test suite and repository formatting checks will also run through CI.

Closes #4854

@jaapmarcus
jaapmarcus self-requested a review September 5, 2026 12:51

@jaapmarcus jaapmarcus left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When running even on a test server with 3 users it runs all quite.

When running it on a server with 100 users wil will even run slower...

0.65 sec instead 0.98 sec when I only do 1 user.

cat /usr/local/hestia/data/user/*/web.conf and then phrase it probally a lot faster

Als there is not need to include the api presets for it on default

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is not need of offering those 4 functions separately in api keys as api keys. If you need them you should probally want more wider api keys or have the knowledge to create them self

Comment thread install/upgrade/versions/1.10.5.sh Outdated
fi

# Add the list-all-user-objects API key permission.
cp "$HESTIA/install/common/api/list-all-user-objects" "$HESTIA/data/api/list-all-user-objects"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to include them in the update script

Comment thread func/list.sh Outdated

printf '{"USER":"%s","OBJECTS":%s}\n' "$user" "$objects"
done <<< "$users"
} | jq --indent 4 -s 'map(.USER as $user | .OBJECTS | with_entries(.value = ({ USER: $user } + .value))) | add // {}'

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know why it list it for every user

But cat /usr/local/hestia/user/*/{any}.conf and parse it once is probably faster.

Read user inventories in one awk pass, require user.conf for inventory owners, and retain legacy output fields with intentional mail CSV and field-isolation differences. Add standalone fixtures and installed-command equivalence coverage.
@thomas-berrio
thomas-berrio force-pushed the feature/4854-list-all-user-objects branch from ea8771e to c891996 Compare September 7, 2026 14:54
@thomas-berrio

Copy link
Copy Markdown
Author

Thanks @jaapmarcus for the review. I’ve addressed your feedback:

  • Replaced per-user command calls with one awk pass over the configuration files.
  • Removed the API preset and upgrade hook.
  • Added standalone fixtures and integration checks comparing the outputs with the existing commands.
  • Added a lightweight CI job for the fixtures.

Two output differences are intentional and documented: missing fields no longer inherit values from previous records, and mail CSV omits the legacy blank separator lines.

Local validation passes: 21 fixture tests, ShellCheck at the CI error threshold, and repository-wide Prettier checks. The full Hestia integration suite still needs CI validation.

Could you take another look and approve the workflows if approval is still required? Thanks!

Use the configured WEBMAIL_ALIAS only when a mail record omits the key. Preserve explicit empty and custom aliases, and test the fallback across records, users and output formats.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Introduce v-list-all-web-domains to list all domains for all users at once

2 participants