Skip to content

Configuration

Shinya Yamaoka edited this page Jan 21, 2020 · 14 revisions

Overview

FrugalOS is configured by two ways below:

  1. Passing command line arguments.
  2. Creates a configuration file and specifies it as a command line argument.

How one can run frugalos with a configuration file

$ frugalos --config-file /path/to/file start

How configuration values are determined

FrugalOS determines configuration values by the following steps:

  1. Reads a configuration file if it is specified.
  2. Overrides the configuration values specified at step 1 by command line arguments.

Configures FrugalOS using a configuration file

FrugalOS can read a YAML file as a configuration file and the format of a configuration file is the following.

The symbol . between strings means that keys are nested. See https://yaml.org/type/index.html if you are not familiar with YAML types.

A list of configuration values

frugalos.data_dir(str)

The path to a directory into where all FrugalOS system data are stored.

frugalos.log_file(str)

The path to a log file.

frugalos.loglevel(str)

One of debug, info, warning, critical and error.

frugalos.max_concurrent_logs(int)

The max size of a queue used in slog. See slog for more details.

frugalos.daemon.executor_threads(int)

The total number of threads used in FrugalOS.

frugalos.daemon.sampling_rate(float)

The rate of sampling jaeger. For example, 0.1 means 10% of all metrics.

frugalos.daemon.stop_waiting_time_millis(int)

How long to wait after receiving a stop request.

frugalos.http_server.bind_addr(str)

The bind address for a HTTP server.

frugalos.rpc_client.tcp_connect_timeout_millis(int)

How long to wait on connecting to a peer before aborting the attempt.

frugalos.rpc_client.tcp_write_timeout_millis(int)

How long to wait on writing data to a peer before aborting the attempt.

frugalos.mds.commit_timeout_threshold(int)

How long to wait after a commit operation before electing a new leader. The actual waiting time is calculated by the following formula:

commit_timeout_threshold * node_polling_interval_millis

frugalos.mds.large_proposal_queue_threshold(int)

The max length of a proposal queue. Re-election of a leader begins if the length of a proposal queue becomes longer than large_proposal_queue_threshold.

frugalos.mds.large_leader_waiting_queue_threshold(int)

The max length of a leader waiting request queue. Re-election of a leader begins if the length becomes longer than large_leader_waiting_queue_threshold.

frugalos.mds.leader_waiting_timeout_threshold(int)

How long to wait before aborting waiting requests and electing a new leader. The actual waiting time is calculated by the following formula:

leader_waiting_timeout_threshold * node_polling_interval_millis

frugalos.mds.node_polling_interval_millis(int)

The interval how often a frugalos_mds node runs periodic tasks.

frugalos.mds.reelection_threshold(int)

The max number of polling counts before electing a new leader. The actual waiting time is calculated by the following formula:

reelection_threshold * node_polling_interval_millis

frugalos.mds.snapshot_threshold_min(int)

The lower bound of taking a node snapshot. A frugalos_mds node chooses an actual snapshot_threshold between snapshot_threshold_min <= N <= snapshot_threshold_max.

frugalos.mds.snapshot_threshold_max(int)

The upper bound of taking a node snapshot. A frugalos_mds node chooses an actual snapshot_threshold between snapshot_threshold_min <= N <= snapshot_threshold_max.

frugalos.mds.log_leader_absence(bool)

Outputs logs if log_leader_absence is true and a MDS node doesn't know a leader node. The logs are periodically generated according to frugalos.mds.log_leader_absence_threshold.

frugalos.mds.log_leader_absence_threshold(int)

How long to wait before logging leader absence in MDS. The actual waiting time is calculated by the following formula:

log_leader_absence_threshold * node_polling_interval_millis

frugalos.segment.mds_client.put_content_timeout_secs(int)

TBD

frugalos.segment.mds_client.get_request_policy.type('conservative' or 'speculative')

Defines the strategy how frugalos sends requests to MDS(metadata service).

A request will never time out if you set this value to conservative. This is the default value.

A request will time out after it consumes a fixed time(defined by configurations) if you set this value to speculative.

frugalos.segment.mds_client.head_request_policy.type('conservative' or 'speculative')

See get_request_policy for more details.

frugalos.segment.mds_client.get_request_policy.timeout_millis(int)

Request time-out time in milliseconds. Timeout value increases exponentially after each timeout. This configuration affects to the behavior only when you set get_request_policy.type to speculative.

frugalos.segment.mds_client.head_request_policy.timeout_millis(int)

See get_request_policy.timeout_millis for more details.

Example

---
frugalos:
  data_dir: "/tmp/srv1"
  log_file: ~
  loglevel: debug
  max_concurrent_logs: 30
  daemon:
    executor_threads: 3
    sampling_rate: 0.1
    stop_waiting_time_millis: 60000
  http_server:
    bind_addr: "127.0.0.1:2222"
  rpc_client:
    tcp_connect_timeout_millis: 2000
    tcp_write_timeout_millis: 2000
  mds:
    commit_timeout_threshold: 20
    large_proposal_queue_threshold: 250
    large_leader_waiting_queue_threshold: 400
    leader_waiting_timeout_threshold: 12
    node_polling_interval_millis: 200
    reelection_threshold: 48
    snapshot_threshold_min: 100
    snapshot_threshold_max: 200
    log_leader_absence: true
    log_leader_absence_threshold: 100
  segment:
    dispersed_client:
      get_timeout_millis: 4000
    mds_client:
      put_content_timeout_secs: 50
      get_request_policy:
        type: 'speculative'
        timeout_millis: 200

Clone this wiki locally