Finding the closest Apache Software Foundation mirror programatically

Question

For my deployment automation needs, I would like to dynamically and programatically determine the closest Apache Software Foundation mirror since the servers are distributed

Answer 1

The mirror URLs in the page are marked up as <strong>, so you can scrape the page to get the top recommendation like this:

curl 'https://www.apache.org/dyn/closer.cgi' |
  grep -o '<strong>[^<]*</strong>' |
  sed 's/<[^>]*>//g' |
  head -1

Additionally, closer.cgi supports an ?as_json=1 query parameter to provide the same information as JSON. The result has a key of preferred for the closest mirror, as well as http for the alternatives.

Answer 2

There is a more elegant way by using jq:

curl -s 'https://www.apache.org/dyn/closer.cgi?as_json=1' | jq --raw-output '.preferred'

Answer 3

Here is an alternative using python:

curl -s 'https://www.apache.org/dyn/closer.cgi?as_json=1' \
| python -c "import sys, json; print json.load(sys.stdin)['preferred']"