Jenkins Pipeline NotSerializableException: groovy.json.internal.LazyMap

Question

Solved: Thanks to below answer from S.Richmond. I needed to unset all stored maps of the groovy.json.internal.LazyMap type whi

Answer 1

EDIT: As pointed out by @Sunvic in the comments, the below solution does not work as-is for JSON Arrays.

I dealt with this by using JsonSlurper and then creating a new HashMap from the lazy results. HashMap is Serializable.

I believe that this required whitelisting of both the new HashMap(Map) and the JsonSlurper.

@NonCPS
def parseJsonText(String jsonText) {
  final slurper = new JsonSlurper()
  return new HashMap<>(slurper.parseText(jsonText))
}

Overall, I would recommend just using the Pipeline Utility Steps plugin, as it has a readJSON step that can support either files in the workspace or text.

Answer 2

This is the detailed answer that was asked for.

The unset worked for me:

String res = sh(script: "curl --header 'X-Vault-Token: ${token}' --request POST --data '${payload}' ${url}", returnStdout: true)
def response = new JsonSlurper().parseText(res)
String value1 = response.data.value1
String value2 = response.data.value2

// unset response because it's not serializable and Jenkins throws NotSerializableException.
response = null

I read the values from the parsed response and when I don't need the object anymore I unset it.

Answer 3

I want to upvote one of the answer: I would recommend just using the Pipeline Utility Steps plugin, as it has a readJSON step that can support either files in the workspace or text: https://jenkins.io/doc/pipeline/steps/pipeline-utility-steps/#readjson-read-json-from-files-in-the-workspace

script{
  def foo_json = sh(returnStdout:true, script: "aws --output json XXX").trim()
  def foo = readJSON text: foo_json
}

This does NOT require any whitelisting or additional stuff.

Answer 4

The other ideas in this post were helpful, but not quite all I was looking for - so I extracted the parts that fit my need and added some of my own magix...

def jsonSlurpLaxWithoutSerializationTroubles(String jsonText)
{
    return new JsonSlurperClassic().parseText(
        new JsonBuilder(
            new JsonSlurper()
                .setType(JsonParserType.LAX)
                .parseText(jsonText)
        )
        .toString()
    )
}

Yes, as I noted in my own git commit of the code, "Wildly-ineffecient, but tiny coefficient: JSON slurp solution" (which I'm okay with for this purpose). The aspects I needed to solve:

1. Completely get away from the java.io.NotSerializableException problem, even when the JSON text defines nested containers
2. Work for both map and array containers
3. Support LAX parsing (the most important part, for my situation)
4. Easy to implement (even with the awkward nested constructors that obviate @NonCPS)

Answer 5

You can use the following function to convert LazyMap to a regular LinkedHashMap (it will keep the order of original data):

LinkedHashMap nonLazyMap (Map lazyMap) {
    LinkedHashMap res = new LinkedHashMap()
    lazyMap.each { key, value ->
        if (value instanceof Map) {
            res.put (key, nonLazyMap(value))
        } else if (value instanceof List) {
            res.put (key, value.stream().map { it instanceof Map ? nonLazyMap(it) : it }.collect(Collectors.toList()))
        } else {
            res.put (key, value)
        }
    }
    return res
}

... 

LazyMap lazyMap = new JsonSlurper().parseText (jsonText)
Map serializableMap = nonLazyMap(lazyMap);

or better use a readJSON step as noticed in earlier comments:

Map serializableMap = readJSON text: jsonText

Answer 6

The way pipeline plugin has been implemented has quite serious implications for non-trivial Groovy code. This link explains how to avoid possible problems: https://github.com/jenkinsci/pipeline-plugin/blob/master/TUTORIAL.md#serializing-local-variables

In your specific case I'd consider adding @NonCPS annotation to slurpJSON and returning map-of-maps instead of JSON object. Not only the code looks cleaner, but it's also more efficient, especially if that JSON is complex.