Skip to content

Repository files navigation

Obol DVT Cluster Assignment

CloudFormation stack templates:

  • 0_s3.yaml

      Provisions an S3 bucket that stores static files for bootstrapping the monitoring/Ethereum containers.
    
  • 1_vpc.yaml

      Provisions base VPC infrastructure for use by other stacks:
      - VPC
      - Subnets
      - Route Tables
      - Security Groups
      - Load Balancers
      - Target Groups
    
  • 2_efs.yaml

      Provisions EFS infrastructure:
      - EFS file system
      - Multi AZ mount points
      
      This gives us a network file system mount that is shared across the cluster. This persists regardless of the Ethereum/Monitoring stack, so can be used to re-sync across any beacon/execution issues. Segregation concerns are discussed below.
    
  • 3_bootstrapping.yaml

      Provisions an EC2 instance that bootstraps a number of operations to support the installation of the monitoring and Ethereum stacks:
      - Copies S3 static content to the EFS file system
      - Performs the key generation ceremony
    
  • 4_monitoring.yaml

      Deploys the monitoring stack
      - Prometheus
      - Grafana
      - Jaeger
    
  • 5_ethereum.yaml (Ethereum infrastructure)

      Deploys the Ethereum Infrastructure
      - Nethermind execution client (working)
      - Lighthouse beacon node (working)
      - Charon relay (working)
      - Charon Node0 (not working) - Crash loops with the same error in local/aws " Error: lock file fork version not in beacon node fork schedule (probably wrong chain/network)"
    

File system security

All containers are EFS mounted against specific paths for their root, so only data specific to that service is available. Important for the key distribution/ceremony. This could be further bolstered by adding IAM authorisation on top of the existing NFS root mapping, to further restrict access to defined paths in the ECS task role policy.

Current implementation (eu-west-2)

CloudFormation stacks

VPC

VPC resources

EFS

EFS resources

Monitoring

Monitoring resources

Deployed infrastructure

CloudFormation Ethereum Stack EFS Metrics EFS Size Nethermind Logs

Improvements identified along the way

  • Tighter IP ranges. Don't need /24 for a single IP on a subnet.
  • Mount targets should use their own subnets, instead of re-using existing ones.
  • Use ECS service discovery instead of load balancers / or some kind of lambda/route53 event based system to assign current container IPs to fixed records. The use of load balancers is very expensive as a service discovery mechanism.
  • VPN to access monitoring resources on private IPs.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors