Skip to content
GNUWeebPublic

About

Telegram Logger Daemon

Resources

Stars

2 stars

Watchers

0 watching

Forks

Repository files navigation

# tgloggerd

tgloggerd is a Telegram logger daemon that runs on a regular user
account. It is built on top of TDLib and keeps its data in MySQL 9. It
records user messages, message edit history, and profile changes such
as usernames, names, and photos. For groups it also tracks the
administrator list and each admin's privileges, keeping a history of
admin changes. Since a regular user account is not pushed admin changes,
the admin list is fetched when a group is first seen and refreshed by
periodic polling (see TG_ADMIN_POLL_INTERVAL in env/settings.env.example).

# Quick start

Everything runs in Docker. You need only Docker with the Compose plugin;
the image carries the toolchain and the containers compile the source on
first start. The full setup is:

1. Install Docker (with the Compose plugin).

2. Provide credentials and configuration:

       cp .env.example .env
       for f in env/*.env.example; do cp "$f" "${f%.example}"; done
       chmod 600 env/*.env
       # edit env/telegram-api.env: TG_API_ID and TG_API_HASH (from
       # https://my.telegram.org); change every password in env/ and
       # set WEB_APP_KEY (env/README).

3. Start MySQL. On its first start it creates the databases and users
   from env/ (the daemon's tgloggerd database and user, plus the web
   app's tgloggerd_web database and its web_ro / web_app users):

       docker compose up -d mysql

4. Log in to Telegram once, interactively. This builds the daemon (the
   first build compiles TDLib and takes a while), applies the daemon
   migrations, and prompts on stdin for your phone number, the login
   code, and your password (if two-step verification is enabled). The
   session is saved under data/users, so later starts do not prompt:

       docker compose run --rm tgloggerd

   Once it is logged in and running, stop it with Ctrl-C.

5. Build and start everything (the daemon and the web interface):

       docker compose up -d

The web interface is then reachable at its private address on the tgnet
network, http://10.0.88.4:8080 (it is not published to the host; put a
reverse proxy in front of it, see web/README.web). Create its first
superadmin (prompts for the password twice):

       docker compose exec web web/build/tgloggerd_web --seed-admin admin

# How the build works

The daemon and the web app share one toolchain image, built from
`Dockerfile` (Ubuntu with the compiler, the MySQL client the daemon's
connector needs, and golang-migrate). The image contains no project
source: docker-compose.yml bind-mounts the repository into each
container, and `run.sh` initializes the vendored git submodules, runs
`cmake`, applies the pending migrations, and execs the binary. Because
the source and the `build/` output are bind-mounted, editing the source
and re-running `docker compose up` recompiles incrementally without
rebuilding the image.

`run.sh <daemon|web|discord|webapi>` is the entrypoint for every
service; the compose file selects the mode per service. The daemon and
web modes apply their migrations on every start, which is idempotent
(`migrate ... up` is a no-op once applied).

# Configuration

All configuration is environment-based. Each service gets its
environment from the files in `env/` (env/README): `env/settings.env`
holds every setting that is not a secret, and each secret has a file
of its own, given only to the services that need it, so a service's
environment holds no other service's secrets. The `env/*.env.example`
files document every variable and its default; the `env/*.env` files
are git-ignored because they hold secrets. `.env` keeps only
docker-compose.yml's own settings (TG_UID, TG_GID). The daemon
variables are prefixed `TG_`, the web variables `WEB_`, and the MySQL
image is configured through `MYSQL_`. A deployment that still has
everything in one `.env` splits it once with `scripts/split-env.py`.

# Project structure

    src/                       Main tgloggerd source code.
      main.c                   Entry point; reads configuration from the
                               environment and starts the daemon.
      log.c, log.h             Small logging helper (pr_info, pr_err, ...).
      mysql/                   Generic MySQL layer (no tgloggerd specifics):
        ConnectionPool.{hpp,cpp}  Thread-safe pool of JDBC connections.
        Database.{hpp,cpp}        Prepared-statement executor over the pool.
      tgloggerd/               The tgloggerd application code:
        TgLoggerd.{hpp,cpp}       Main class; wires TDLib to the database.
        TDLib.{hpp,cpp}           Thin wrapper around TDLib (see below).
        DB.{hpp,cpp}              DB class lifecycle and facade interface.
        tgloggerd_extern.{h,cpp}  C ABI used by main.c.
        helpers/                  C helpers (logging, filesystem).
        models/                   Data models (.hpp) and their SQL query
                                  implementations (.cpp) (User, File, ...).
    migrations/                golang-migrate SQL migrations for the schema.
                               Tables that store a messaging platform's data are
                               prefixed per platform (telegram_*, discord_*), and
                               tables that RELATE two platforms carry both names,
                               source first (telegram_discord_webhooks,
                               discord_telegram_routes). See the naming rules
                               below and web/docs/db-naming.md before adding
                               tables.
    submodules/                Third-party dependencies vendored as git
                               submodules: TDLib (td), MySQL Connector/C++
                               (mysql-connector-cpp), Drogon, and MariaDB
                               Connector/C (the last two for the web app).
    data/                      Runtime data (git-ignored contents):
      users/                     Per-account TDLib data directories.
      mysql/                     MySQL server data (docker volume).
      storage/files/             Downloaded files, named by SHA-256.
    Dockerfile                 Toolchain image for both components.
    docker-compose.yml         mysql, tgloggerd, web, discordd and webapi
                               services.
    docker/mysql/init/         MySQL first-init hook (web database/users).
    run.sh                     Container entrypoint: build, migrate, run.
    .env.example               Compose's own settings; copy to .env.
    env/                       The services' settings and secrets.
    web/                       Optional server-rendered web interface for
                               browsing the database (a separate Drogon
                               application; see web/README.web).
    webapi/                    The Go web service that is replacing the
                               Drogon web, serving the JSON API under /api/
                               (see webapi/README).

# Database table naming

tgloggerd is growing from a Telegram-only logger into a hub that bridges
several messaging platforms, so table names have to say which platform --
or which pair of platforms -- they belong to. Two rules, and the full
version with examples lives in web/docs/db-naming.md:

1. A table holding data for ONE platform is prefixed with that platform:

       telegram_users, telegram_group_messages, telegram_files
       discord_guilds, discord_channels, discord_messages

2. A table that RELATES two platforms is prefixed with BOTH, source first,
   then destination:

       telegram_discord_webhooks       config: mirror Telegram -> Discord
       telegram_discord_sent_messages  what that forwarder posted
       discord_telegram_routes         config: mirror Discord -> Telegram
       discord_telegram_sent_messages  what that forwarder sent

   The direction is part of the name. telegram_discord_* and
   discord_telegram_* are different tables describing opposite flows, and
   the prefix order is the only thing distinguishing them. A third
   platform follows the same shape: slack_telegram_routes, and so on.

   A single prefix on a bridge table is misleading. discord_webhooks --
   renamed to telegram_discord_webhooks in migration 000020 -- read as
   Discord-only data even though every row is keyed by a Telegram chat,
   and it left no room to name the reverse direction once one existed.

   Only tables that genuinely relate two platforms get two prefixes:
   discord_endpoints (interned Discord webhook URLs) and telegram_bots
   (Telegram bot credentials) each describe one platform, so they keep a
   single prefix even though the bridge uses them.

The convention is schema-only. HTTP routes, search-API entity keys and
JSON response keys stay platform-neutral and unchanged.

# Interaction with TDLib

TDLib ships very large headers that noticeably slow tgloggerd's
compilation, so the code that touches TDLib is kept to a minimum. All
direct TDLib usage lives in src/tgloggerd/TDLib.cpp, which exposes a
small wrapper (tgloggerd::TDLib) through TDLib.hpp.

# License

tgloggerd is free software under the GNU General Public License, either
version 2 or (at your option) any later version. Both license texts ship
with the tree: COPYING is version 2, COPYING.GPLv3 is version 3. Source
files carry an SPDX-License-Identifier: GPL-2.0-or-later tag; the
migrations, templates and build files do not yet, and are covered by
LICENSE, which also records the two copyright holders and the terms of
the third-party code the build links.

The "or later" is doing real work rather than following convention. The
daemon links OpenSSL 3.x, which is Apache-2.0, and Apache-2.0 cannot be
combined with version 2 alone -- its patent-termination and
indemnification clauses are the "further restrictions" GPLv2 section 6
forbids. Version 3 accommodates them, so a recipient who takes the work
under version 3 has a combination that is consistent; under
GPL-2.0-only no such path existed. The choice is the recipient's, per
GPLv2 section 9, so in practice the source tree is GPL-2.0-or-later
while any shipped binary that links OpenSSL is GPL-3.0-or-later. See
LICENSE for what that implies, including the one third-party exception
the arrangement now depends on, and two things it does not fix.

About

Telegram Logger Daemon

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages