Shell script - Sorting 'AWS cloudwatch metrics' json array based on the “Timestamp” property value which comes in ISO 8601 UTC format

Question

I have an Amazon cloudwatch ELB Latency metrics like below.

{
"Datapoints": [
    {
        "Timestamp": "2016-10-18T12:11:00Z",
        "Average": 0.25880099632013942,
        "Minimum": 0.00071811676025390625,
        "Maximum": 3.2039437294006352,
        "Unit": "Seconds"
    },
    {
        "Timestamp": "2016-10-18T12:10:00Z",
        "Average": 0.25197337517680762,
        "Minimum": 0.00063610076904296875,
        "Maximum": 2.839790821075439,
        "Unit": "Seconds"
    },
    {
        "Timestamp": "2016-10-18T12:19:00Z",
        "Average": 0.2287127116954388,
        "Minimum": 0.00061678886413574219,
        "Maximum": 1.416410446166992,
        "Unit": "Seconds"
    }
 ]

}

i'm running 'awscli' inside shell script for getting this , but data is not returned in chronological order and timestamp is in ISO 8601 UTC format. I need to sort this array based on the Timestamp to get the data in chronological order.

My Aim: I have one more cloudwatch metrics data from ELB RequestCount metrics like below.

{
"Datapoints": [
    {
        "Timestamp": "2016-10-18T12:11:00Z",
        "Sum": 217732.0,
        "Unit": "Count"
    },
    {
        "Timestamp": "2016-10-18T12:15:00Z",
        "Sum": 227120.0,
        "Unit": "Count"
    },
  ]

}

I was looking to sort these metrices based on the timestamp and create a match between the latency and number of request at each timestamp. Also , i have to calculate the time difference between the starttime and endtime which might not be possible from the format that is received here.

I'm using shell script and cannot figure out a way to so this. Any help would be really appreciated. TIA

Answer 1

JMESPATH has a sort_by method that can be used for this - here is just an example

aws cloudwatch get-metric-statistics \
  --metric-name CPUUtilization \
  --start-time 2016-10-01T23:18:00 --end-time 2016-10-19T23:18:00 --period 3600 \
  --namespace AWS/EC2 --statistics Maximum \
  --dimensions Name=InstanceId,Value=<YOURINSTANCE> \
  --query 'sort_by(Datapoints,&Timestamp)[*]'

If want to go with jq instead you'll use jq's sort_by method as follow

aws cloudwatch get-metric-statistics \
  --metric-name CPUUtilization \
  --start-time 2016-10-01T23:18:00 --end-time 2016-10-19T23:18:00 --period 3600 \
  --namespace AWS/EC2 --statistics Maximum \
  --dimensions Name=InstanceId,Value=<YOURINSTANCE> \
| jq -r '.Datapoints | sort_by(.Timestamp)[]'

If you're more a bash guy, you can use linux sort command

aws cloudwatch get-metric-statistics \
  --metric-name CPUUtilization \
  --start-time 2016-10-01T23:18:00 --end-time 2016-10-19T23:18:00 --period 3600 \
  --namespace AWS/EC2 --statistics Maximum \
  --dimensions Name=InstanceId,Value=<YOURINSTANCE> \
  --output text \
| sort -k 3

the dates are in the 3rd column so you will sort this column(-k 3)