Repository navigation
Folders and files
| Name | Name | Last commit date | ||
|---|---|---|---|---|
Repository files navigation
# tgloggerd
tgloggerd is a Telegram logger daemon that runs on a regular user
account. It is built on top of TDLib and keeps its data in MySQL 9. It
records user messages, message edit history, and profile changes such
as usernames, names, and photos. For groups it also tracks the
administrator list and each admin's privileges, keeping a history of
admin changes. Since a regular user account is not pushed admin changes,
the admin list is fetched when a group is first seen and refreshed by
periodic polling (see TG_ADMIN_POLL_INTERVAL in env/settings.env.example).
# Quick start
Everything runs in Docker. You need only Docker with the Compose plugin;
the image carries the toolchain and the containers compile the source on
first start. The full setup is:
1. Install Docker (with the Compose plugin).
2. Provide credentials and configuration:
cp .env.example .env
for f in env/*.env.example; do cp "$f" "${f%.example}"; done
chmod 600 env/*.env
# edit env/telegram-api.env: TG_API_ID and TG_API_HASH (from
# https://my.telegram.org); change every password in env/ and
# set WEB_APP_KEY (env/README).
3. Start MySQL. On its first start it creates the databases and users
from env/ (the daemon's tgloggerd database and user, plus the web
app's tgloggerd_web database and its web_ro / web_app users):
docker compose up -d mysql
4. Log in to Telegram once, interactively. This builds the daemon (the
first build compiles TDLib and takes a while), applies the daemon
migrations, and prompts on stdin for your phone number, the login
code, and your password (if two-step verification is enabled). The
session is saved under data/users, so later starts do not prompt:
docker compose run --rm tgloggerd
Once it is logged in and running, stop it with Ctrl-C.
5. Build and start everything (the daemon and the web interface):
docker compose up -d
The web interface is then reachable at its private address on the tgnet
network, http://10.0.88.4:8080 (it is not published to the host; put a
reverse proxy in front of it, see web/README.web). Create its first
superadmin (prompts for the password twice):
docker compose exec web web/build/tgloggerd_web --seed-admin admin
# How the build works
The daemon and the web app share one toolchain image, built from
`Dockerfile` (Ubuntu with the compiler, the MySQL client the daemon's
connector needs, and golang-migrate). The image contains no project
source: docker-compose.yml bind-mounts the repository into each
container, and `run.sh` initializes the vendored git submodules, runs
`cmake`, applies the pending migrations, and execs the binary. Because
the source and the `build/` output are bind-mounted, editing the source
and re-running `docker compose up` recompiles incrementally without
rebuilding the image.
`run.sh <daemon|web|discord|webapi>` is the entrypoint for every
service; the compose file selects the mode per service. The daemon and
web modes apply their migrations on every start, which is idempotent
(`migrate ... up` is a no-op once applied).
# Configuration
All configuration is environment-based. Each service gets its
environment from the files in `env/` (env/README): `env/settings.env`
holds every setting that is not a secret, and each secret has a file
of its own, given only to the services that need it, so a service's
environment holds no other service's secrets. The `env/*.env.example`
files document every variable and its default; the `env/*.env` files
are git-ignored because they hold secrets. `.env` keeps only
docker-compose.yml's own settings (TG_UID, TG_GID). The daemon
variables are prefixed `TG_`, the web variables `WEB_`, and the MySQL
image is configured through `MYSQL_`. A deployment that still has
everything in one `.env` splits it once with `scripts/split-env.py`.
# Project structure
src/ Main tgloggerd source code.
main.c Entry point; reads configuration from the
environment and starts the daemon.
log.c, log.h Small logging helper (pr_info, pr_err, ...).
mysql/ Generic MySQL layer (no tgloggerd specifics):
ConnectionPool.{hpp,cpp} Thread-safe pool of JDBC connections.
Database.{hpp,cpp} Prepared-statement executor over the pool.
tgloggerd/ The tgloggerd application code:
TgLoggerd.{hpp,cpp} Main class; wires TDLib to the database.
TDLib.{hpp,cpp} Thin wrapper around TDLib (see below).
DB.{hpp,cpp} DB class lifecycle and facade interface.
tgloggerd_extern.{h,cpp} C ABI used by main.c.
helpers/ C helpers (logging, filesystem).
models/ Data models (.hpp) and their SQL query
implementations (.cpp) (User, File, ...).
migrations/ golang-migrate SQL migrations for the schema.
Tables that store a messaging platform's data are
prefixed per platform (telegram_*, discord_*), and
tables that RELATE two platforms carry both names,
source first (telegram_discord_webhooks,
discord_telegram_routes). See the naming rules
below and web/docs/db-naming.md before adding
tables.
submodules/ Third-party dependencies vendored as git
submodules: TDLib (td), MySQL Connector/C++
(mysql-connector-cpp), Drogon, and MariaDB
Connector/C (the last two for the web app).
data/ Runtime data (git-ignored contents):
users/ Per-account TDLib data directories.
mysql/ MySQL server data (docker volume).
storage/files/ Downloaded files, named by SHA-256.
Dockerfile Toolchain image for both components.
docker-compose.yml mysql, tgloggerd, web, discordd and webapi
services.
docker/mysql/init/ MySQL first-init hook (web database/users).
run.sh Container entrypoint: build, migrate, run.
.env.example Compose's own settings; copy to .env.
env/ The services' settings and secrets.
web/ Optional server-rendered web interface for
browsing the database (a separate Drogon
application; see web/README.web).
webapi/ The Go web service that is replacing the
Drogon web, serving the JSON API under /api/
(see webapi/README).
# Database table naming
tgloggerd is growing from a Telegram-only logger into a hub that bridges
several messaging platforms, so table names have to say which platform --
or which pair of platforms -- they belong to. Two rules, and the full
version with examples lives in web/docs/db-naming.md:
1. A table holding data for ONE platform is prefixed with that platform:
telegram_users, telegram_group_messages, telegram_files
discord_guilds, discord_channels, discord_messages
2. A table that RELATES two platforms is prefixed with BOTH, source first,
then destination:
telegram_discord_webhooks config: mirror Telegram -> Discord
telegram_discord_sent_messages what that forwarder posted
discord_telegram_routes config: mirror Discord -> Telegram
discord_telegram_sent_messages what that forwarder sent
The direction is part of the name. telegram_discord_* and
discord_telegram_* are different tables describing opposite flows, and
the prefix order is the only thing distinguishing them. A third
platform follows the same shape: slack_telegram_routes, and so on.
A single prefix on a bridge table is misleading. discord_webhooks --
renamed to telegram_discord_webhooks in migration 000020 -- read as
Discord-only data even though every row is keyed by a Telegram chat,
and it left no room to name the reverse direction once one existed.
Only tables that genuinely relate two platforms get two prefixes:
discord_endpoints (interned Discord webhook URLs) and telegram_bots
(Telegram bot credentials) each describe one platform, so they keep a
single prefix even though the bridge uses them.
The convention is schema-only. HTTP routes, search-API entity keys and
JSON response keys stay platform-neutral and unchanged.
# Interaction with TDLib
TDLib ships very large headers that noticeably slow tgloggerd's
compilation, so the code that touches TDLib is kept to a minimum. All
direct TDLib usage lives in src/tgloggerd/TDLib.cpp, which exposes a
small wrapper (tgloggerd::TDLib) through TDLib.hpp.
# License
tgloggerd is free software under the GNU General Public License, either
version 2 or (at your option) any later version. Both license texts ship
with the tree: COPYING is version 2, COPYING.GPLv3 is version 3. Source
files carry an SPDX-License-Identifier: GPL-2.0-or-later tag; the
migrations, templates and build files do not yet, and are covered by
LICENSE, which also records the two copyright holders and the terms of
the third-party code the build links.
The "or later" is doing real work rather than following convention. The
daemon links OpenSSL 3.x, which is Apache-2.0, and Apache-2.0 cannot be
combined with version 2 alone -- its patent-termination and
indemnification clauses are the "further restrictions" GPLv2 section 6
forbids. Version 3 accommodates them, so a recipient who takes the work
under version 3 has a combination that is consistent; under
GPL-2.0-only no such path existed. The choice is the recipient's, per
GPLv2 section 9, so in practice the source tree is GPL-2.0-or-later
while any shipped binary that links OpenSSL is GPL-3.0-or-later. See
LICENSE for what that implies, including the one third-party exception
the arrangement now depends on, and two things it does not fix.