背景

公司内部有个包大小检测程序,能检测两个包的大小然后输出 html 报告。但缺少一个预警通知的机制,希望通过 jenkins pipeline 加上

问题

为了解析 html 报告,用上了 groovy 自带的 XmlSlurper 库,这个库可以解析 xml dom 元素,然后再提取其中指定的元素出来。这个库网上搜得到的基本用法:

def rootNode = new XmlSlurper().parseText(
    '<root><one a1="uno!"/><two>Some text!</two></root>' )

 assert rootNode.name() == 'root'

所以参考这个用法,对应在 pipeline 里写了相关的处理解析语句:

...
def htmlStr = sh(script: "cat diffReport.html", returnStdout: true)
def rootNode = new XmlSlurper().parseText(htmlStr)
def diffSizeStr = rootNode.table[0].tbody.tr.td[2].toString()
...
//后面是解析并判断大小,然后打印是否比阈值大的信息

此时,使用起来没有任何问题,比较和打印(println)是否比阈值大也没任何问题

然后申请到权限后,末尾多加一个调用 http 接口发送企业微信机器人通知的功能:

...
def htmlStr = sh(script: "cat diffReport.html", returnStdout: true)
def rootNode = new XmlSlurper().parseText(htmlStr)
def diffSizeStr = rootNode.table[0].tbody.tr.td[2].toString()
...
//省略解析并判断大小,然后打印是否比阈值大的信息的代码

// 发送通知
sh 'curl -X POST -d '{"message": "包大小检测不通过,包大小差异为 '+diffSizeStr+' "}' http://xx.xx/sendMessage/'

此时,运行后开始报错:

an exception which occurred:
    in field com.cloudbees.groovy.cps.impl.BlockScopeEnv.locals
    in object com.cloudbees.groovy.cps.impl.BlockScopeEnv@14f316d8
    in field com.cloudbees.groovy.cps.impl.ProxyEnv.parent
    in object com.cloudbees.groovy.cps.impl.BlockScopeEnv@487dc4d4
    in field com.cloudbees.groovy.cps.impl.CallEnv.caller
    in object com.cloudbees.groovy.cps.impl.FunctionCallEnv@12c500d9
    in field com.cloudbees.groovy.cps.Continuable.e
    in object org.jenkinsci.plugins.workflow.cps.SandboxContinuable@5d6dce53
    in field org.jenkinsci.plugins.workflow.cps.CpsThread.program
    in object org.jenkinsci.plugins.workflow.cps.CpsThread@5bac1141
    in field org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.threads
    in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@56d65f4b
    in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@56d65f4b
Caused: java.io.NotSerializableException: groovy.util.slurpersupport.NodeChild
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:926)
    at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)
    at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)
    at org.jboss.marshalling.MarshallerObjectOutputStream.writeObjectOverride(MarshallerObjectOutputStream.java:50)
    at org.jboss.marshalling.river.RiverObjectOutputStream.writeObjectOverride(RiverObjectOutputStream.java:179)
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:344)
    at java.util.HashMap.internalWriteEntries(HashMap.java:1790)
    at java.util.HashMap.writeObject(HashMap.java:1363)
    at sun.reflect.GeneratedMethodAccessor34.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.jboss.marshalling.reflect.JDKSpecific$SerMethods.callWriteObject(JDKSpecific.java:156)
    at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:191)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1028)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
    at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)
    at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)
    at org.jboss.marshalling.MarshallerObjectOutputStream.writeObjectOverride(MarshallerObjectOutputStream.java:50)
    at org.jboss.marshalling.river.RiverObjectOutputStream.writeObjectOverride(RiverObjectOutputStream.java:179)
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:344)
    at java.util.TreeMap.writeObject(TreeMap.java:2438)
    at sun.reflect.GeneratedMethodAccessor53.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.jboss.marshalling.reflect.JDKSpecific$SerMethods.callWriteObject(JDKSpecific.java:156)
    at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:191)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1028)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
    at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:58)
    at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:111)
    at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverWriter.lambda$writeObject$0(RiverWriter.java:144)
    at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:241)
    at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverWriter.writeObject(RiverWriter.java:143)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:500)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:476)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgramIfPossible(CpsThreadGroup.java:463)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:387)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:93)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:259)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:247)
    at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
    at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
    at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

排查过程

找到的第一个结果: https://stackoverflow.com/questions/41171550/jenkins-java-io-notserializableexception-groovy-util-slurpersupport-nodechild

回答里的答案,基本大意是说把出问题的这部分代码放到函数里,函数前面加个 @NonCPS 注解就好了

所以对应改了下代码:

...
def htmlStr = sh(script: "cat diffReport.html", returnStdout: true)
def rootNode = new XmlSlurper().parseText(htmlStr)
def diffSizeStr = rootNode.table[0].tbody.tr.td[2].toString()
...
//省略解析并判断大小,然后打印是否比阈值大的信息的代码
sendMessage(diffSizeStr)

// 发送通知
@NonCPS
def sendMessage(String diffSizeStr) {
    sh 'curl -X POST -d '{"message": "包大小检测不通过,包大小差异为 '+diffSizeStr+' "}' http://xx.xx/sendMessage/'
}

结果还是报错,问题没有解决

后面几个结果,基本都是说变量使用了类似 def name = rootNode.name() 的写法,改为 def name = rootNode.name().toString() 解决。但我这里代码本身写法就加了 toString() ,所以排除这个原因

猜测是不是 toString() 不够彻底,导致有些东西还是引入进来了。所以试着不使用解析出来的字符串,改为别的字符串

...
def htmlStr = sh(script: "cat diffReport.html", returnStdout: true)
def rootNode = new XmlSlurper().parseText(htmlStr)
def diffSizeStr = rootNode.table[0].tbody.tr.td[2].toString()
...
//省略解析并判断大小,然后打印是否比阈值大的信息的代码
sendMessage('100KB')

// 发送通知
@NonCPS
def sendMessage(String diffSizeStr) {
    sh 'curl -X POST -d '{"message": "包大小检测不通过,包大小差异为 '+diffSizeStr+' "}' http://xx.xx/sendMessage/'
}

结果竟然还是报错。难道是这个接口返回结果有问题?换个接口试试:

...
def htmlStr = sh(script: "cat diffReport.html", returnStdout: true)
def rootNode = new XmlSlurper().parseText(htmlStr)
def diffSizeStr = rootNode.table[0].tbody.tr.td[2].toString()
...
//省略解析并判断大小,然后打印是否比阈值大的信息的代码
sendMessage('100KB')

// 发送通知
@NonCPS
def sendMessage(String diffSizeStr) {
    sh 'curl http://www.baidu.com'
}

结果竟然还是报错。中途还尝试了各种姿势,比如单引号改双引号啥的,均以失败告终。

看回错误信息 java.io.NotSerializableException: groovy.util.slurpersupport.NodeChild ,这里有两个信息:

1、异常名称是 NotSerializableException ,和序列化有关

2、引起异常的类是 groovy.util.slurpersupport.NodeChild ,和 sluper 这个类有关

难道,和我 curl 没关系?回头看前面已经写好的代码:

...
def htmlStr = sh(script: "cat diffReport.html", returnStdout: true)
def rootNode = new XmlSlurper().parseText(htmlStr)
def diffSizeStr = rootNode.table[0].tbody.tr.td[2].toString()
...

这里第二个变量,rootNode 看起来符合无法序列化的特征。难道是这里的问题?那把这个变量去掉试试,反正也没啥用:

...
def htmlStr = sh(script: "cat diffReport.html", returnStdout: true)
def diffSizeStr = new XmlSlurper().parseText(htmlStr).table[0].tbody.tr.td[2].toString()
...

再次运行,错误消失了!

解决方案总结

def rootNode = new XmlSlurper().parseText(htmlStr)
def diffSizeStr = rootNode.table[0].tbody.tr.td[2].toString()

合并成

def diffSizeStr = new XmlSlurper().parseText(htmlStr).table[0].tbody.tr.td[2].toString()

就解决了。

原因解析及猜测

首先可以确定的是,出问题的原因,是 new XmlSlurper().parseText(htmlStr) 返回的对象无法序列化,所以导致序列化时会报错。

但为啥之前没加 curl 命令的时候没出错?个人猜测,原因很可能是因为这里调用了 shell 。

jenkins pipeline 有一个特性,就是 groovy 的变量可以直接在 shell 里面以 ${var} 的形式直接引用。但 shell 毕竟是另一个运行环境,无法直接传入 groovy 对象,所以可能这里有做了一次序列化,把对象转为字符串之类的基本数据类型方便传入 shell 。而之前没加 curl 前,后面都没有调用过 shell ,所以没有触发序列化。

佐证:

新写了一个 pipeline

node {
    def rootNode = new XmlSlurper().parseText("<xml></xml>")
    println "success"
}

直接运行,没有报错。增加一个 shell 调用

node {
    def rootNode = new XmlSlurper().parseText("<xml></xml>")
    sh 'echo "shell"'
    println "success"
}

直接报错

总结

1、排查过程还是切忌猜疑,要从日志和错误信息出发。这次有点天马行空了,实际效率更低了。

2、看搜索结果要细看,不要只看自己觉得要看的 “重点” 。之前第一次的搜索结果,另一条结果 https://stackoverflow.com/questions/58771874/xmlslurper-in-jenkins-pipeline-how-to-avoid-java-io-notserializableexception?rq=1 里面有提及不建议在 pipeline 里传递 XmlSlurper().parseText() 生成的对象


↙↙↙阅读原文可查看相关链接,并与作者交流