Command Line

Usage: python -m gatelogue_aggregator [OPTIONS] COMMAND [ARGS]...

Options:
  --version   Show the version and exit.
  -h, --help  Show this message and exit.

Commands:
  run     actually run the aggregator
  schema  export a JSON schema of the current data format

Run

Usage: python -m gatelogue_aggregator run [OPTIONS]

  actually run the aggregator

Options:
  --cache-dir PATH                where to cache files downloaded from the
                                  Internet (preferably a temporary directory)
                                  [default: /tmp/gatelogue]
  --timeout INTEGER               how long to wait for a network request in
                                  seconds before aborting and failing
                                  [default: 60]
  --cooldown INTEGER              how long to wait before sending new requests
                                  to the same URL if `429 Too Many Requests`
                                  is received  [default: 15]
  -o, --output PATH               file to output the result to, in JSON
                                  [default: data.json]
  -f, --fmt / -F, --no-fmt        prettify the JSON result  [default: F]
  -r, --report / -R, --no-report  print a report of all nodes after merger
                                  [default: r]
  -g, --graph PATH                file to output a graph representation of all
                                  nodes and objects to, in SVG
  -w, --max_workers INTEGER       maximum number of concurrent workers that
                                  download and process data  [default: 8]
  -ce, --cache-exclude TEXT       re-retrieve data for these sources instead
                                  of loading from cache (separate with `;`,
                                  use `*` for all sources)  [default: ""]
  -i, --include TEXT              sources to retrieve from (do not use with
                                  --exclude) (separate with `;`, use `*` for
                                  all sources)  [default: ""]
  -e, --exclude TEXT              sources NOT to retrieve from (do not use
                                  with --include) (separate with `;`, use `*`
                                  for all sources)  [default: ""]
  -h, --help                      Show this message and exit.

Schema

Usage: python -m gatelogue_aggregator schema [OPTIONS]

  export a JSON schema of the current data format

Options:
  -o, --output PATH     file to output the result to, in JSON  [default:
                        data.json]
  -f, --fmt / --no-fmt  prettify the JSON result  [default: no-fmt]
  -h, --help            Show this message and exit.