Skip to content

librity/ft_netmon

BranchesTags

Repository files navigation

42 São Paulo - Labs, 3rd Edition

42 São Paulo License Code size in bytes Lines of code Top language Last commit

Build Norminette v3

A network monitoring service with pthreads in pure C. 

/-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-\
|                                                                                     |
|                                              ::^~^:::     ::                        |
|                                   :^~~JPGPJ5GB#&&&&&#GYJ?!: :    :                  |
|                              ::!YB#&&@@@@@@@@@@@@@@@@@@@@@#GPY7~^:  :               |
|                         ^!?YP#&@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@&B5!:               |
|                     :!Y#&@&&@@@@@@@@@@@@@&&&##BBBBGBBB&&&&@@@@@@@@@&5?~:            |
|                :!JYB&@@@@@@@@@@@@@@@&&#??77^~^  :     :::^~!JP5G&@@@&#B?^           |
|            ^?P#&@@@@@@@@@@@@@@@&&BYPGJPY?PP?!?Y~!^^^:::::       :!?P#@@&BY^         |
|          Y#&@@@@@@@@@@@@@@@&#BGBPBG##P&##&##5B&GB5?YY?5!7~~:^:::     ~5B&&P~:       |
|        !&@@@@@@@@@@@@@@&&#&B&&#&@@@&@&@&@&@@@&&&&#&&#B#GYY5??~~~^~  :   ~?#&P^      |
|        Y@@@@@@@@@@@&&G&B#&&@&GPYJ7!!!7!!7?Y5PGB#&&&&&&&&#&B#BP557!~^!!:    ?&#?     |
|       :^?B##&&#BB5GBB#@@&#5YY~^:  :~J: :~   :!~::~7JPB&@&@&&&&GBPPPY?J!!^:  :YB~    |
|      :~~!?JJJY555P#&@@&BGGY5J?^^~?5&@&B#&&PPB&&J^7?: :^^~JG#&&&###P#PG5J???^:  ~7   |
|    ^^!J7555BBG&&&@@@#5PBBY?75B&@@@@@@@@@@@@@@@&&@#5GP~    :~?G#@#&GGPGP5G57~::      |
|    ^~7GJPP#B&B&&@&&BPGP5J!P&@@&#@@@&@@@@@@@@@@@@@&@@&BJ!~!~   :!5#BGP5GPPP~~:       |
|    !??PGBG&G#5&@@#&##B5!P@@&#P!P@@@@@@@@@@@@&&&@@BGB#&@@@&Y!~!^  :!55?YJ7~:         |
|   !??PBG5GG#&&@#7!5G55J#@&B?!^ ?@@@@#G@@@@@&&&@@@!::~!5P#&@@&PPY~                   |
|   :~?J5G#G#&&5!   :JPB#@&57:   :B@@@B5@@@@&&&&@@5:      :75BBPY!                    |
|    :!7YPBG#G:    ?@&J^:P5~      :P&@@@@@&@@@@@#?           ^~!J~     :~!^!::        |
|     :^?J5YJY:    :!?7YJ?7^:       ^YB&&@@@&#P7      ::~!!?JJ!:   :^!J?Y?7!::        |
|       : ::    :^^^:P#&##5BPP??|^:    :^~~~^   ^^~~77JJ5J7^     ^75PJ5YY~^^          |
|           :^!7BB&#G&BB&G#PGJYGGGP555P5JYYY555GPJ?!!~:    ::^J55PBGP??|^:            |
|                 :^7~PJBBG#&@&&@B&GJGJP?^~~~7~!7!~^:^7: ::^~~!~?GPB#5P5P?~           |
|            :^^~J7YYBG#B#&B&&&&##&#BB5JY?5J?~7!^?5BGG&BB#G&#GJJ!^^:                  |
|               :^!7!JY5YBYG5GBG&G#&#&##&#B&B#BGG###BGBP5YY^^:                        |
|                      ^ ~:!^77?YJYPJ5G5P55YGP?57PPJJ7?~^^::                          |
|                            : :^:::: ^~:!^^^ :                                       |
|                                                                                     |
|-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-|
|                                                                                     |
!               ░░░    ░░ ░░░░░░░ ░░░░░░░░ ░░░    ░░░  ░░░░░░  ░░░    ░░              !
!               ▒▒▒▒   ▒▒ ▒▒         ▒▒    ▒▒▒▒  ▒▒▒▒ ▒▒    ▒▒ ▒▒▒▒   ▒▒              !
:               ▒▒ ▒▒  ▒▒ ▒▒▒▒▒      ▒▒    ▒▒ ▒▒▒▒ ▒▒ ▒▒    ▒▒ ▒▒ ▒▒  ▒▒              :
:               ▓▓  ▓▓ ▓▓ ▓▓         ▓▓    ▓▓  ▓▓  ▓▓ ▓▓    ▓▓ ▓▓  ▓▓ ▓▓              :
.               ██   ████ ███████    ██    ██      ██  ██████  ██   ████              .
.                                                            by lpaulo-m              .
.                                                                                     .
.                          __ __     ___ __  __     __                                :
:                         /  /  \|\ | | |__)/  \|  (_                                 :
:                         \__\__/| \| | | \ \__/|____)                                :
!                                                                                     !
!                         - CONTROL+Q: Exits program cleanly.                         !
!                         - CONTROL+C: Exits program.                                 !
|                                                                                     |
\-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-/

📜 Table of Contents

🧐 About

This is a network monitor that periodically sends request and saves the results to a .log file. It currently handles HTTP, HTTPS, PING and DNS requests.

It can be configured im multiple ways with the netmon.db file. Each line in the file specifies a target, protocol, and request frequency (period between requests). Columns are tab-separated and must conform to the format:

====================================================================================================
| PROTOCOL | ROWS (tab separated)                                                                  |
====================================================================================================
| HTTP     | name    HTTP     address    http_method    expected_http_code    frequency_in_seconds |
| HTTPS    | name    HTTPS    address    http_method    expected_http_code    frequency_in_seconds |
| PING     | name    PING     address    frequency_in_seconds                                      |
| DNS      | name    DNS      address    frequency_in_seconds        dns_server_ipv4               |
====================================================================================================

The program parses the file and spawns a scheduler thread for each target that periodically enqueues a new request in the request pool. It also spawns worker threads that dequeue and perform the requests.

The worker then prints a simplified result of the request to STDOUT and saves all the relevant data to the specified .log file.

Usage Gif

It can also parse the log file and print it to STDOUT in simple form with the flag --simplify.

The thread pool implementation is based on CodeVault's video:

I handled HTTPS request with libcurl, and made my own implementation of HTTP, PING and DNS based on:

This program uses Raw Sockets for the PING implementation and must be executed with root privileges (or any other user with CAP_NET_ADMIN capability).

🏁 Getting Started

⚙️ Prerequisites

To compile and run this program you will need:

  • A shell and a C compiler (gcc ideally).
  • GNU Make installed in your computer.

You will also need to install libcurl as a dev dependency. You can get it through apt-get (recommended):

$ sudo apt-get install libcurl4-openssl-dev

Or you can install it manually:

$ wget https://curl.se/download/curl-7.84.0.zip
$ unzip curl-7.84.0.zip
$ cd curl-7.84.0
$ ./configure --with-openssl --prefix=$HOME/.local
$ make
$ make install

🖥️ Installing

Clone the repo and build with make:

$ git clone --recurse-submodules https://github.com/42sp/ft_netmon.git ft_netmon
$ cd ft_netmon
$ make

It should create a netmon executable that you can run with sudo ./netmon.

🏳️ Options and Flags

  • -h and --help: Print the help message.
  • -s and --simplify: Parse the .log file and print it to STDOUT in simple form.
  • -d and --debug: Verbose, prints debug throughout runtime.
  • --config-file <file>: Specify the config file path (default ./netmon.db).
  • --log-file <file>: Specify the log file path (default ./netmon.log).
  • --workers <number>: Set the initial number of worker threads (default 8).

✅ Checklist

Mandatory

  • Follows norminette 3.3.51
  • Makefile rules: $(NAME) all clean fclean re
  • Compile an executable named netmon.
  • Compiles with -Wall -Wextra -Werror
  • Should not quit unexpectedly (segmentation fault, bus error, double free, etc.)
  • All allocated heap memory properly freed, no memory leaks.
    • Check memory leaks with valgrind
  • Allowed functions:
    • Anything you want, as long as you can reasonably justify it.
  • Handle flags
    • -h and —help display help message and exits cleanly.
    • -s and —simplify reads the log file, prints it in simple form to STDOUT and exits cleanly.
    • -d and —debug flags that print debug info through runtime.
    • —config-file <file> Set the config file path (default ./netmon.db)
    • --log-file <file> Set the log file path (default ./netmon.log)
    • --workers <number> Set the initial number of worker threads (default 8).
  • Validate config file netmon.db:
    • Handle comments with #.
    • Validate HTTP configs:
      • Should have 6 tab-separated fields
      • Validate HTTP_METHOD.
      • Validate expected HTTP_CODE.
      • Validate timeout in seconds (unsigned int > 0).
    • Validate PING configs:
      • Should have 4 tab-separated fields
      • Validate timeout in seconds (unsigned int > 0).
    • Validate DNS configs:
      • Should have 5 tab-separated fields
      • Validate timeout in seconds (unsigned int > 0).
      • Validate IPv4 address
  • Parse configs for each protocol:
    • Parse HTTP targets
    • Parse PING targets
    • Parse DNS targets
  • Each monitoring service/request must use the configurations defined in netmon.db.
  • Handle request with a Thread Pool
    • Spawn workers that:
      • Dequeue task safely (mutex and cond)
      • Perform network requests
      • Display the simplified result to STDOUT.
      • Save all relevant request data to log file in parsable form.
    • Spawn a scheduler for every target in config file:
      • Enqueue target request safely (mutex and cond)
      • Enqueue every timeout seconds.
    • All threads exit cleanly and join main thread on CONTROL+Q with no deadlocks.
      • Enable tty raw mode and manually handle key presses.
  • Requests:
    • HTTP
      • Get IP with getaddrinfo()
      • My own implementation with sys/socket.h
      • Save all relevant request data to log file in parsable form.
    • HTTPS
      • Use libcurl for the entire thing
      • Save all relevant request data to log file in parsable form.
    • PING
      • Get IP with getaddrinfo()
      • My own implementation with sys/socket.h
      • Save all relevant request data to log file in parsable form.
    • DNS
      • My own implementation with sys/socket.h
      • Save all relevant request data to log file in parsable form.
  • netmon.log should contain all available request data.
  • netmon.log should be saved in parsable form, such that it can be displayed simply with the --simplify flag.
  • Should work as an useful and professional network monitoring service.
  • Clean and readable code.
  • Add a beautiful README.md explaining how the program works.
  • Come up with simple and creative solutions.
  • Follow specs as much as possible.

Bonus

  • Parse add extra flags
    • Add help message.
    • Set custom config path.
    • Set custom log path.
  • Take special care with optimizations, code quality and design patterns.
  • Use external tools to plan and track various stages of development.
  • Notify bad request through Discord, email, Slack or Webhook.
  • Thoroughly test the application:
    • Flag tests
    • Config tests
    • Thread Pool tests
    • Request handler tests
    • Logger tests
  • Identifies odd behaviors in target services, like and increase in latency.
  • Aggregate analysis of netmon.log data, with diagrams and graphs on the CLI.
  • Handle MQTT and TCP requests/monitoring:
    • TCP
      • My own implementation with sys/socket.h
      • Save all relevant request data to log file in parsable form.
    • MQTT
      • Use libcurl for the entire thing
      • Save all relevant request data to log file in parsable form.
  • Supervisor threads:
    • Restart worker and scheduler threads that exit with an error.

🛸 42 São Paulo

Part of the larger 42 Network, 42 São Paulo is a software engineering school that offers a healthy alternative to traditional education:

  • It doesn't have any teachers and classes.
  • Students learn by cooperating and correcting each other's work (peer-to-peer learning).
  • Its focus is as much on social skills as it is on technical skills.
  • It's completely free to anyone that passes its selection process - The Piscine

It's an amazing school, and I'm grateful for the opportunity.

📚 Resources

HTTP and HTTPS

HTTP Methods

HTTP Headers

HTTP Implementations

PING

PING Implementations

DNS

DNS Implementations

DNS Libs

MQTT

TCP

[getaddrinfo](https://gist.github.com/jirihnidek/bf7a2363e480491da72301b228b35d5d#file-getaddrinfo_example-c)()

arpa/inet.h

sockaddr_in

sys/socket.h

Time to Live

CAP_NET_ADMIN and SOCK_RAW

sys/capability.h

setsockopt()

pthread.h

pthread_create

**pthread_detach**()

pthread_mutex_t

pthread_cond_t

pthread_exit()

**pthread_cancel**()

_pthread_key_create_()

Request Scheduling Options

  • while/sleep
  • timer_create
  • Wild threads
  • Thread Pool + Thread Schedulers + Thread Supervisor

Thread Pools

Thread-safe time.h

time_t vs. clock_t

strftime

Thread-safe read(), write() and printf()

timer_create

Wild threads

Thread Supervisor

libcurl

# LIBCURL WORKSPACE INSTALL
$ wget https://curl.se/download/curl-7.84.0.zip
$ unzip -d curl-7.84.0.zip
$ ./configure --with-openssl --prefix=$HOME/.local
$ make
$ make install
# Lib in ~/.local/lib
# Includes in ~/.local/include
$ curl-config --cflags
$ curl-config --libs
  • HTTP
  • HTTPS
  • MQTT

Mongoose

Paho MQTT Client

Mosquito

Doxygen style documentation

URLs

lalloc - Listed Memory Allocation

A linked list in the control structure with all the allocated memory pointers. The interface function ft_lalloc allocates memory and adds the pointer to the list. The interface function ft_free_lalloc frees all pointers and the list.

realloc

Git submodule

Log Formats

$ bat /var/log/syslog

Github Actions libcurl

DNS Servers

VSCode

C JSON Libs

json-c

Mock Server

Multi-threaded Web Server

C Quirks

C Libs

regex.h

Testing

ASCII Art

Services

GIF

About

42 São Paulo - Labs, 3rd Edition - netmon - A Threaded Network Monitoring Service in Pure C

www.42sp.org.br/

Topics

c dns http https ping http-client pthreads network-monitoring threads norminette 42born2code 42 42school threadpool monitoring-tool https-client 42projects 42saopaulo

Resources

Readme

License

MIT license
Activity

Stars

9 stars

Watchers

2 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Contributors 2

  •  
  •  

Languages