Branch	Status
master

• Table of contents
• Description
• Install
• External Ressources Dependencies
  • Postgresql Port Forwarding
  • S3 Vector Bucket Access
• Docker
  • Docker build locally
  • Docker run locally
  • Push the locally built docker image to ECR
• Unit Testing
  • Prerequisites
  • Starting the tests
• Download WMS image legends
• Python Code Styling
• Updating Python Packages
• OTEL
  • Environment variables
• Varia
  • Lint a JSON file
• Local Smoke test for Open Telemetry
  • Make a HTTP Query

Next generation services https://api3.geo.admin.ch for https://map.geo.admin.ch In mid August 2022 the project has been migrated to python3, docker and eu-central-1.

The required environment variables are set in .env.default. They can be adapted or you can use a copy of .env.default, e.g. .env.mine and use that instead.

Install the python virtual environment (still virtualenvat this point)

make setup

Build the Pylons settings files and run the local waitress server

summon -p ssm make serve

You may want to customize the variables. Copy the file .env.default as .ven.mine, change the variables you want and use them with

summon -p ssm make ENV_FILE=.env.mine serve

📖 You need some external ressource to run the service, see External ressources dependencies

To run the service locally you need to have access to the following external ressources:

• Postgresql database pg-geodata-replica.bgdi.ch
• S3 bucket service-mf-chsdi3-grid-geojsons-dev-swisstopo (NOTE: this bucket is only required by some of the Identity endpoints)

You can use the ssh port forwarding feature to have access to pg-geodata-replica.bgdi.ch by using the jump host:

ssh ssh0a.prod.bgdi.ch -L 5432:pg-geodata-replica.bgdi.ch:5432

Then set the DBHOST environment variable to localhost (you can do this in your own environment file e.g. .env.mine and run the make file as follow: summon -p ssm make ENV_FILE=.env.mine serve)

or simpler, when you use the ssh_config provided by infra-ansible-bgdi-dev:

ssh jumphost-pg-geodata-replica

To have access to the S3 bucket, you can either set your AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variable. ⚠️ When using those variables make sure that the command are not saved into history file !

Alternatively if you are using zsh you can use the aws plugin (see oh-my-zsh aws plugin) with the following command:

acp swisstopo-bgdi-dev

This command will automatically use your AWS profile swisstopo-bgdi-dev for any AWS connection services.

make dockerbuild

summon -p ssm make dockerrun

📖 You need some external ressource to run the service, see External ressources dependencies

First log in to the AWS ECR registry with:

make dockerlogin

afterwards you can push the locally built image to ECR with:

make dockerpush

• PostgreSQL DB pg-geodata-replica.bgdi.ch must be reachable
• Access to AWS services
  • Read access to S3 bucket service-mf-chsdi3-grid-geojsons-dev-swisstopo (can be disable with S3_TESTS=0)

See External Ressources Dependencies for more infos on those prerequisites.

To run the tests enter

summon -p ssm make test

Or if you use your own environment file

summon -p ssm make ENV_FILE=.env.mine test

⚠️ If you don't have AWS Access you can disable the S3 tests as follow

summon -p ssm make S3_TESTS=0 test

In order to download all images of a layer in the correct format and with the correct dimensions, simply use:

make legends BODID=ch.layername WMSHOST=wms.geo.admin.ch

Alternatively, you can also download a WMS legend for a specific scale.

make legends BODID=ch.layername WMSHOST=wms.geo.admin.ch WMSSCALELEGEND=1000

You will need the optipgn tool order to download the legends, use sudo apt install optipng to install it.

We are currently using the FLAKES 8 convention for Python code. You can find more information about our code styling here:

• http://www.python.org/dev/peps/pep-0008/
• http://pep8.readthedocs.org/en/latest/index.html

You can find additional information about autopep8 here:

• https://pypi.python.org/pypi/autopep8/

To check the code styling:

make lint

To autocorrect most linting mistakes

make autolint

All packages used in production are pinned to a major version. Automatically updating these packages will use the latest minor (or patch) version available. Packages used for development, on the other hand, are not pinned unless they need to be used with a specific version of a production package (for example, boto3-stubs for boto3).

To update the packages to the latest minor/compatible versions, run:

pipenv update --dev

To see what major/incompatible releases would be available, run:

pipenv update --dev --outdated

To update packages to a new major release, run:

pipenv install logging-utilities~=5.0

OpenTelemetry instrumentation can be done in many different ways, from fully automated zero-code instrumentation (otel-operator) to purely manual instrumentation. We use the so called OTEL programmatical instrumentation approach where we import the specific instrumentation libraries and initialize them with the instrument() method of each library.

The following env variables can be used to configure OTEL

export PATH=$(npm bin):$PATH
jsonlint-cli --pretty temp.json > chsdi/static/vectorStyles/ch.meteoschweiz.messwerte-foehn-10min.json

1. (optional) aws --profile swisstopo-bgdi-dev sso login
2. ssh jumphost-pg-geodata-replica to setup SSH Tunnel to DB
3. docker compose up to run a local Jaeger server
4. summon -p ssm make dockerrun

Open in a browser or curl:

http://localhost:8009/rest/services/ech/MapServer/identify?layers=all:ch.bav.haltestellen-oev&sr=2056&geometry=2596144.500865812,1192821.252700392&mapExtent=2595220.51,1191795.99,2597416.51,1193957.99&imageDisplay=1098,1081,96&geometryFormat=geojson&geometryType=esriGeometryPoint&limit=10&tolerance=10&returnGeometry=true&lang=en