Description

From the original issue #3831:

The orderbook(s) and the autopilot run as separate processes but both need access to native prices. Unfortunately they don't share the same underlying native price cache which can lead to inconsistencies - especially for niche tokens.
For example the orderbook could be able to fetch a native price and allow an order to be placed. Then the autopilot will pick up the order from the DB and try to fetch the native price as well. If a token is very niche there might only be 1 solver supporting it and it's possible that it serves the orderbook request correctly but rate limits the autopilot request.

To fix this we need to have 1 native price cache that all services can use. Ideally we'd move all the quoting logic into a separate service and make it use a Redis cache (in case we need to scale the service horizontally).
However that is a lot of work and we can get a pretty reasonably approximation with a lot less effort.

For that we just need to make the orderbook forward the native price requests to the autopilot. This will then cause the autopilot cache to be the 1 shared cache for all the processes.

Changes

• Added axum server to autopilot to handle /native_price requests
• The orderbook forwards requests to a configurable URL
  • The new functionality is integrated into the current native price estimator config in the following way:
    • Driver native price estimator now has the Driver prefix to distinguish between legacy and new approach
    • The Forwarder prefix is added to use the Forwarder native price estimator.
  • The orderbook also forwards the native price timeout. This query param is optional, and autopilot fallbacks to the configured one if not provided in the request.
• Adjusted e2e tests
  • Autopilot is now used as a native price estimator in all the tests.
  • The dual_autopilot_only_leader_produces_auctions requires running 2 autopilots at the same time for some period of time. Since it doesn't make much sense to specify more than 2 autopilot URLs in the orderbook config, a reverse proxy is introduced to make this test pass. This proxy simulates k8s behavior: when 2 instances of the same service are running, requests are automatically routed between them. An alternative approach is to run 2 autopilot APIs on different ports and specify them in the orderbook config, but the reverse proxy simulates the production behavior.
• The API server is initialized before the metrics for a reason. Currently, the metrics port is used as a readiness probe in the k8s cluster. To avoid introducing an additional readiness probe endpoint, I decided to continue using the metrics port. But if the metrics readiness probe becomes available before the API server, the new service is added to the k8s proxy pool and starts serving orderbook requests, but since the server is not initialized fully, there is a chance that some of the requests fail. Initializing the metrics only after the API server resolves this issue.
• An openapi.yml file with the hope that this component will be decentralized in the future.

How to test

Updated e2e tests. This also requires running tests on staging manually.

Related Issues

Partially fixes #3831

Further implementation

Once we are confident the new approach works well, the old config with native price estimators can be dropped from the orderbook crate.