using awk sed to parse update puppet file

Question

I have a puppet file with a number of lines of code that has a section that looks like this:

$defaultrepo=myrepo
$defaultbranch=mybranch

gitmod::pullstuff {'othergitcode':
      gitcommit => "b54123be540adrwer3b65872384e0101c5f94c926b81",
      gitorg      => "${defaultrepo}",
      branch    => "${defaultbranch}",
    }

gitmod::pullstuff {'mygitcode':
      gitcommit => "b54123be540adrfer3b65872384e0101c5f94c926b81",
      gitorg      => 'awesomerepo',
      branch    => "master",
    }

It can have any number of blocks that will look similar to the two blocks of code listed above.

I am looking for a way to parse the file and pull out the values as variables then use them to obtain the lastest git commit and update the value. I was thinking of awk and sed in a bash script so that it can do something like this. e.g.

values could be: $gitrepo=mygitcode $gitorg=awesomerepo $gitbranch=master

for i in $(awk "regexto pull out each block from  gitmod::pullstuff \{'whatever': through         to the ending \}" puppetfile.pp);
do
    $gitrepo="sed/awk to pullout git repo"
    $gitorg="sed/awk to pullout git org"
    $gitbranch="sed/awk to pullout git branch"
    $newcommit=git ls-remote https://github.com/$gitorg/$gitrepo.git refs/heads/$gitbranch
    sed -i "replace commit with $newcommit"
done

I am not a developer by trade so please bear with me. Could someone please point out what the general structure should look like. Don't worry about the regex etc I can mess around and figure that out. Just wondering how I can get it so that it treats each "block" it finds individually. For some reason I can't get awk to separate them out properly and it messes up the formatting.

Like I can do something like this:

awk "/gitmod::pullstuff/{a=4}{if(a-->0){print;next}}"

and it will pull out everything I want. What I need is to somehow stick it into a for statement so that I can treat each instance that awk finds as a separate block that I can do stuff too.

Answer 1

You can try something like:

awk -f m.awk puppetfile.pp  puppetfile.pp

where m.awk is:

NR==FNR {
    if (/^gitmod/) {
        gitrepo=getRepo()
        getline
        getline
        gitorg=getOrg()
        getline
        branch=getBranch()
        com[++i]=getNewCommit()
    }
    else if (/^\$[[:alnum:]]*=/) {
        vn=getVarName()
        val=getVarValue()
        var[vn]=val
    }
    next
}

/^gitmod/ {
    print
    getline
    sub(/".*"/,"\""com[++j]"\"")
}
{ print }

function getVarValue(a) {
    match($0,/=([[:alnum:]]+)[[:blank:]]*/,a)
    return a[1]
}

function getVarName(a) {
    match($0,/\$([[:alnum:]]+)=/,a)
    return "${"a[1]"}"
}

function getNewCommit(cmd,var) {
    cmd="ls-remote https://github.com/"gitorg"/"gitrepo".git refs/heads/"branch
    cmd |& getline var
    return var
}

function getBranch(a,br) {
    match($0,/"(.*)"/,a)
    br=a[1]
    if (br in var) br=var[br]
    return br
}

function getOrg(a,org) {
    match($0,/"(.*)"/,a)
    org=a[1]
    if (org in var) org=var[org]
    return org
}
function getRepo(a,rep) {
    match($0,/\{"(.*)":/,a)
    rep=a[1]
    if (rep in var) rep=var[rep]
    return rep
}

Answer 2

Here is a different take on your problem. If you store all this data in a YAML file :

gits:
  othergitcode:
    gitcommit: "b54123be540adrwer3b65872384e0101c5f94c926b81"
    gitorg:    "myreop",
    branch:    "mybranch",
  mygitcode:
    gitcommit: "b54123be540adrfer3b65872384e0101c5f94c926b81",
    gitorg:    'awesomerepo',
    branch:    "master"

Then you can have it on puppet with something like this:

create_resources('gitmod::pullstuff', hiera('gits'))

And this make it easy to manipulate in any scripting language (except awk, I suppose this answer falls short here ...).