In May 2024 I was contacted by a headhunter looking to hire a Python Engineer for the Man Group. In order to prepare for an interview I spent a few days studying the tech stack that was listed in the job description [3]. Nothing really came of the interview, but I did come up with a possible overall architecture diagram (see below) and found out that Man AHL are largely open source. This repo is a place to put stuff as I learn more about their tech stack.
The folowing technologies have been mentioned in connection to Man Group in various open sources:
| Component | Source | Description | Logo | Notes |
|---|---|---|---|---|
| ENVIRONMENT | ||||
| Bare Metal | 2. | Hardware | ||
| Linux | 5. | OS | ||
| OpenStack | 5. | Cloud IaaS. Basically, soft for provisioning bare metal into cloud resources | ||
| Prometheus | 5. | Server monitoring soft | Mentioned in close association with Grafana in source 1. | |
| Grafana | 5. | Data visualisation soft | Mentioned in close association with Prometheus in source 1. Probably used to visualise output from Prometheus for continuous server monitoring | |
| INFRASTRUCTURE | ||||
| Docker | 5. | Containerisation soft, allows to deploy standardised images of other softs hardware agnostically | ||
| Kubernetes | 5. | Container management soft | ||
| Ansible | 5. | Infra as code, provisioning soft | ||
| DATASOURCE | ||||
| RMDS/TREP | 1. | Reuters | Mentioned in close association with Kafka in source 1. | |
| Kafka | 5. | Event logger | Mentioned in close association with RMDS/TREP in source 1. Probably used to log tick data into files | |
| STORAGE | ||||
| ArcticDB | 4. | Man's in-house open source data storage solution | Mentioned in close association with MongoDB in source 1. | |
| MongoDB | 5. | File based DB | Mentioned in close association with ArcticDB in source 1. | |
| Oracle | 5. | Relational DB | ||
| MINING | ||||
| Elasticsearch | Indexing/search engine | Infering that Man uses this from ELKDocker, as it looks like a fitting solution to index datafiles in Mongo | ||
| Spark | 1. | Analytics engine for large-scale data | Probably both Apache Spark and PySpark | |
| Airflow | 5. | Workflow management soft for data engineering pipelines | ||
| PROGRAMMING | ||||
| Python3 - numpy | 5. | Array maths library | ||
| Python3 - scipy | 5. | Scientific computing library | ||
| Python3 - pandas | 5. | Dataframe manipulation library | ||
| Python3 - scikit-learn | 5. | Machine learning library | ||
| Python3 - TensorFlow | 5. | Machine learning library | ||
| Java | 5. | General purpose programming language | ||
| C/C++ | 1. | General purpose programming languages | ||
| Bash | General purpose scripting languages | Unconfirmed, but Airfow uses bash to integrate with non-python scripts and source 4. mentions scirpting | ||
| CI/CD | ||||
| Jenkins | 5. | Development pipeline automation soft | ||
| Bitbucket | 5. | Repository | ||
| Jira | Issue management soft | Unconfirmed, but BitBucket has tight integration with Jira and source 4. mentions agile | ||
| TBD | ||||
| ELKDocker | 1. | Is this deviantony/docker-elk ? |
In order to get a better grasp on the technologies listed above, I have compiled a list of learning resources, available here.
- JD – Python Engineer – Market Data Platform.pdf (non-public)
- PyData: James Blackburn - Python and MongoDB as a Platform for Financial Market Data, last accessed 2024-06-15, https://www.youtube.com/watch?v=FVyIxdxsyok
- LinedIn_Ad_15-06-2024 14-09-47.png
- ArcticDB homepage, last accessed 2024-06-15, https://arcticdb.io/
- LinedIn_Ad_19-06-2024 20-39-27.png