公司内部有个包大小检测程序,能检测两个包的大小然后输出 html 报告。但缺少一个预警通知的机制,希望通过 jenkins pipeline 加上
为了解析 html 报告,用上了 groovy 自带的 XmlSlurper 库,这个库可以解析 xml dom 元素,然后再提取其中指定的元素出来。这个库网上搜得到的基本用法:
def rootNode = new XmlSlurper().parseText(
'<root><one a1="uno!"/><two>Some text!</two></root>' )
assert rootNode.name() == 'root'
所以参考这个用法,对应在 pipeline 里写了相关的处理解析语句:
...
def htmlStr = sh(script: "cat diffReport.html", returnStdout: true)
def rootNode = new XmlSlurper().parseText(htmlStr)
def diffSizeStr = rootNode.table[0].tbody.tr.td[2].toString()
...
//后面是解析并判断大小,然后打印是否比阈值大的信息
此时,使用起来没有任何问题,比较和打印(println)是否比阈值大也没任何问题
然后申请到权限后,末尾多加一个调用 http 接口发送企业微信机器人通知的功能:
...
def htmlStr = sh(script: "cat diffReport.html", returnStdout: true)
def rootNode = new XmlSlurper().parseText(htmlStr)
def diffSizeStr = rootNode.table[0].tbody.tr.td[2].toString()
...
//省略解析并判断大小,然后打印是否比阈值大的信息的代码
// 发送通知
sh 'curl -X POST -d '{"message": "包大小检测不通过,包大小差异为 '+diffSizeStr+' "}' http://xx.xx/sendMessage/'
此时,运行后开始报错:
an exception which occurred:
in field com.cloudbees.groovy.cps.impl.BlockScopeEnv.locals
in object com.cloudbees.groovy.cps.impl.BlockScopeEnv@14f316d8
in field com.cloudbees.groovy.cps.impl.ProxyEnv.parent
in object com.cloudbees.groovy.cps.impl.BlockScopeEnv@487dc4d4
in field com.cloudbees.groovy.cps.impl.CallEnv.caller
in object com.cloudbees.groovy.cps.impl.FunctionCallEnv@12c500d9
in field com.cloudbees.groovy.cps.Continuable.e
in object org.jenkinsci.plugins.workflow.cps.SandboxContinuable@5d6dce53
in field org.jenkinsci.plugins.workflow.cps.CpsThread.program
in object org.jenkinsci.plugins.workflow.cps.CpsThread@5bac1141
in field org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.threads
in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@56d65f4b
in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@56d65f4b
Caused: java.io.NotSerializableException: groovy.util.slurpersupport.NodeChild
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:926)
at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)
at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)
at org.jboss.marshalling.MarshallerObjectOutputStream.writeObjectOverride(MarshallerObjectOutputStream.java:50)
at org.jboss.marshalling.river.RiverObjectOutputStream.writeObjectOverride(RiverObjectOutputStream.java:179)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:344)
at java.util.HashMap.internalWriteEntries(HashMap.java:1790)
at java.util.HashMap.writeObject(HashMap.java:1363)
at sun.reflect.GeneratedMethodAccessor34.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.jboss.marshalling.reflect.JDKSpecific$SerMethods.callWriteObject(JDKSpecific.java:156)
at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:191)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1028)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)
at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)
at org.jboss.marshalling.MarshallerObjectOutputStream.writeObjectOverride(MarshallerObjectOutputStream.java:50)
at org.jboss.marshalling.river.RiverObjectOutputStream.writeObjectOverride(RiverObjectOutputStream.java:179)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:344)
at java.util.TreeMap.writeObject(TreeMap.java:2438)
at sun.reflect.GeneratedMethodAccessor53.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.jboss.marshalling.reflect.JDKSpecific$SerMethods.callWriteObject(JDKSpecific.java:156)
at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:191)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1028)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:58)
at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:111)
at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverWriter.lambda$writeObject$0(RiverWriter.java:144)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:241)
at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverWriter.writeObject(RiverWriter.java:143)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:500)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:476)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgramIfPossible(CpsThreadGroup.java:463)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:387)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:93)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:259)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:247)
at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
java.io.NotSerializableException: groovy.util.slurpersupport.NodeChild
回答里的答案,基本大意是说把出问题的这部分代码放到函数里,函数前面加个 @NonCPS 注解就好了
所以对应改了下代码:
...
def htmlStr = sh(script: "cat diffReport.html", returnStdout: true)
def rootNode = new XmlSlurper().parseText(htmlStr)
def diffSizeStr = rootNode.table[0].tbody.tr.td[2].toString()
...
//省略解析并判断大小,然后打印是否比阈值大的信息的代码
sendMessage(diffSizeStr)
// 发送通知
@NonCPS
def sendMessage(String diffSizeStr) {
sh 'curl -X POST -d '{"message": "包大小检测不通过,包大小差异为 '+diffSizeStr+' "}' http://xx.xx/sendMessage/'
}
结果还是报错,问题没有解决
后面几个结果,基本都是说变量使用了类似 def name = rootNode.name()
的写法,改为 def name = rootNode.name().toString()
解决。但我这里代码本身写法就加了 toString()
,所以排除这个原因
猜测是不是 toString()
不够彻底,导致有些东西还是引入进来了。所以试着不使用解析出来的字符串,改为别的字符串
...
def htmlStr = sh(script: "cat diffReport.html", returnStdout: true)
def rootNode = new XmlSlurper().parseText(htmlStr)
def diffSizeStr = rootNode.table[0].tbody.tr.td[2].toString()
...
//省略解析并判断大小,然后打印是否比阈值大的信息的代码
sendMessage('100KB')
// 发送通知
@NonCPS
def sendMessage(String diffSizeStr) {
sh 'curl -X POST -d '{"message": "包大小检测不通过,包大小差异为 '+diffSizeStr+' "}' http://xx.xx/sendMessage/'
}
结果竟然还是报错。难道是这个接口返回结果有问题?换个接口试试:
...
def htmlStr = sh(script: "cat diffReport.html", returnStdout: true)
def rootNode = new XmlSlurper().parseText(htmlStr)
def diffSizeStr = rootNode.table[0].tbody.tr.td[2].toString()
...
//省略解析并判断大小,然后打印是否比阈值大的信息的代码
sendMessage('100KB')
// 发送通知
@NonCPS
def sendMessage(String diffSizeStr) {
sh 'curl http://www.baidu.com'
}
结果竟然还是报错。中途还尝试了各种姿势,比如单引号改双引号啥的,均以失败告终。
看回错误信息 java.io.NotSerializableException: groovy.util.slurpersupport.NodeChild
,这里有两个信息:
1、异常名称是 NotSerializableException ,和序列化有关
2、引起异常的类是 groovy.util.slurpersupport.NodeChild ,和 sluper 这个类有关
难道,和我 curl 没关系?回头看前面已经写好的代码:
...
def htmlStr = sh(script: "cat diffReport.html", returnStdout: true)
def rootNode = new XmlSlurper().parseText(htmlStr)
def diffSizeStr = rootNode.table[0].tbody.tr.td[2].toString()
...
这里第二个变量,rootNode 看起来符合无法序列化的特征。难道是这里的问题?那把这个变量去掉试试,反正也没啥用:
...
def htmlStr = sh(script: "cat diffReport.html", returnStdout: true)
def diffSizeStr = new XmlSlurper().parseText(htmlStr).table[0].tbody.tr.td[2].toString()
...
再次运行,错误消失了!
def rootNode = new XmlSlurper().parseText(htmlStr)
def diffSizeStr = rootNode.table[0].tbody.tr.td[2].toString()
合并成
def diffSizeStr = new XmlSlurper().parseText(htmlStr).table[0].tbody.tr.td[2].toString()
就解决了。
首先可以确定的是,出问题的原因,是 new XmlSlurper().parseText(htmlStr)
返回的对象无法序列化,所以导致序列化时会报错。
但为啥之前没加 curl 命令的时候没出错?个人猜测,原因很可能是因为这里调用了 shell 。
jenkins pipeline 有一个特性,就是 groovy 的变量可以直接在 shell 里面以 ${var}
的形式直接引用。但 shell 毕竟是另一个运行环境,无法直接传入 groovy 对象,所以可能这里有做了一次序列化,把对象转为字符串之类的基本数据类型方便传入 shell 。而之前没加 curl 前,后面都没有调用过 shell ,所以没有触发序列化。
佐证:
新写了一个 pipeline
node {
def rootNode = new XmlSlurper().parseText("<xml></xml>")
println "success"
}
直接运行,没有报错。增加一个 shell 调用
node {
def rootNode = new XmlSlurper().parseText("<xml></xml>")
sh 'echo "shell"'
println "success"
}
直接报错
1、排查过程还是切忌猜疑,要从日志和错误信息出发。这次有点天马行空了,实际效率更低了。
2、看搜索结果要细看,不要只看自己觉得要看的 “重点” 。之前第一次的搜索结果,另一条结果 https://stackoverflow.com/questions/58771874/xmlslurper-in-jenkins-pipeline-how-to-avoid-java-io-notserializableexception?rq=1 里面有提及不建议在 pipeline 里传递 XmlSlurper().parseText()
生成的对象