接口测试 Java、Python、Node.js 在接口测试过程中如何快速进行 diff 测试?

qianxing · August 07, 2018 · Last by 果冻 replied at March 18, 2019 · 3010 hits

背景

在接口测试过程中,当所测接口存在多个版本时,经常遇到需要对新旧接口返回值进行对比测试(diff测试)。

方法

方法1. 将新旧接口返回值以文件存储,然后利用文件对比、json对比的工具逐一比较进行测试。
方法2. 自动化测试,使用python、javascript、java中断言进行对比。

java:junit+fastjson,使用Assert.assertEquals进行对比测试

  • 1. 示例代码
package com.test.apitest;

import com.alibaba.fastjson.JSON;
import org.junit.*;
import java.util.Map;

public class JunitJsonDiffTest {

@Test
public void test_json_diff_sub_nodes(){
String oldJson = "{\"user\":{\"name\":\"test\",\"age\":10},\"hello\":\"world\",\"success\":true}";
String newJson = "{\"success\":true,\"hello\":\"world\",\"user\":{\"name\":\"test\",\"age\":10}}";

Object oldJsonVo = JSON.parseObject(oldJson);
Object newJsonVo = JSON.parseObject(newJson);
// json字段顺序虽然不一样,但是对比结果相同
Assert.assertEquals(oldJsonVo,newJsonVo);
}

@Test
public void test_json_with_date(){
String oldJson = "{\"hello\":\"你好\",\"success\":true,\"birthDay\":\"1995-06-15 10:05:00\"}";
String newJson = "{\"hello\":\"你好\",\"success\":true,\"birthDay\":\"1995-06-15 20:05:00\"}";

Object oldJsonVo = JSON.parseObject(oldJson);
Object newJsonVo = JSON.parseObject(newJson);
// 对比过程中需要考虑是否支持中文
Assert.assertEquals(oldJsonVo,newJsonVo);
}

}
  • 2. 对比效果

  • 3. 使用mvn命令生成html报告
    pom.xml文件中增加maven-surefire-report依赖项:

<dependency>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-report-plugin</artifactId>
<version>2.22.0</version>
</dependency>

生成html报告命令:

mvn clean  surefire-report:report
  • 4. html报告

python:unitest+json,使用assertDictEqual进行对比测试

  • 1. 示例代码
import json

class TestDictCompare(unittest.TestCase):
def test_jsondiff_same(self):
self.maxDiff=None
oldJson = json.loads('{"title":"com.tencent.news","fullTitle":"Androbugs report com.tencent.news","timedOut":false,"duration":0,"state":"passed","speed":"fast","pass":true,"fail":false,"pending":false}')
newJson = json.loads('{"title":"com.tencent.news","fullTitle":"Androbugs report com.tencent.news","timedOut":false,"duration":0,"state":"passed","speed":"fast","pass":true,"fail":false,"pending":false}')
self.assertDictEqual(oldJson,newJson)

def test_jsondiff_different(self):
self.maxDiff=None
oldJson = json.loads('{"title":"com.tencent.news","fullTitle":"Androbugs report com.tencent.news","timedOut":false,"duration":0,"state":"passed","speed":"fast","pass":true,"fail":false,"pending":false}')
newJson = json.loads('{"title":"com.tencent.news2","fullTitle":"Androbugs report com.tencent.news","timedOut":false,"duration":0,"state":"passed","speed":"fast","pass":true,"fail":false,"pending":false}')
self.assertDictEqual(oldJson,newJson)


if __name__ == '__main__':
unittest.main()
  • 2. 运行结果

  • 3. 使用pip安装pytest-html包生成html报告

 pip install pytest-html
pytest --html=sondiff.html test\test_jsondiff.py
  • 4. 生成的html报告

javascript:mocha+chai,使用deepEqual进行对比测试

  • 1. 示例代码
var assert = require("chai").assert;

describe("json diff", function () {
it("json相同", function () {
var oldJson = JSON.parse('{"title":"com.tencent.news","fullTitle":"Androbugs report com.tencent.news","timedOut":false,"duration":0,"state":"passed","speed":"fast","pass":true,"fail":false,"pending":false}')
var newJson = JSON.parse('{"title":"com.tencent.news","fullTitle":"Androbugs report com.tencent.news","timedOut":false,"duration":0,"state":"passed","speed":"fast","pass":true,"fail":false,"pending":false}')
assert.deepEqual(oldJson, newJson)
})

it("json不同", function () {
var oldJson = JSON.parse('{"title":"com.tencent.news","fullTitle":"Androbugs report com.tencent.news","timedOut":false,"duration":0,"state":"passed","speed":"fast","pass":true,"fail":false,"pending":false}')
var newJson = JSON.parse('{"title":"com.tencent.news2","fullTitle":"Androbugs report com.tencent.news","timedOut":false,"duration":0,"state":"passed","speed":"fast","pass":true,"fail":false,"pending":false}')
assert.deepEqual(oldJson, newJson)
})
})
  • 2. 运行结果

  • 3. 使用mochawesome生成html报告

npm install mochawesome --save-dev
mocha test\jsondiff.test.js --reporter mochawesome
  • 4. 生成的html报告

如何快速进行diff测试?

原理:在JAVA、python中将需要对比的接口URL进行参数化,使用一个用例来进行diff测试。在JavaScript中将接口url及其参数放到一个参数数组中,使用数组forEach方法来进行对比测试。

JAVA代码

package com.test.apitest;

import com.alibaba.fastjson.JSON;
import org.junit.Assert;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

import java.util.Map;

import com.github.kevinsawicki.http.HttpRequest;

public class BatchApiTest {
@DataProvider
public Object[][] data() {
Object[][] diffApis = new Object[][]{
{
"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/demo",
"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/demo"
},
{
"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/hello",
"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/hello"
},
{
"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/user",
"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/user"
},
{
"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/1/list",
"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/1/list"
},
{
"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/2/list",
"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/2/list"
},
{
"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/3/list",
"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/3/list"
},
};
return diffApis;
}

@Test(dataProvider = "data")
public void test_apiDiff(String oldApiUrl, String newApiUrl) {
String oldJsonString = HttpRequest.get(oldApiUrl)
.accept("application/json")
.body();
String newJsonString = HttpRequest.get(newApiUrl)
.accept("application/json")
.body();
Map<String,Object> oldJsonVo = JSON.parseObject(oldJsonString);
Map<String,Object> newJsonVo = JSON.parseObject(newJsonString);
Assert.assertEquals(oldJsonVo,newJsonVo);
}
}

Python示例

# -*- coding:utf-8 -*-
import unittest
import requests
from parameterized import parameterized, param
diffApis = [
# diff apis
{
"oldapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/demo",
"newapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/demo"
},
{
"oldapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/hello",
"newapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/hello"
},
{
"oldapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/user",
"newapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/user"
},
{
"oldapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/1/list",
"newapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/1/list"
},
{
"oldapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/2/list",
"newapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/2/list"
},
{
"oldapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/3/list",
"newapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/3/list"
},
]


class TestParameterized(unittest.TestCase):
@parameterized.expand(
[param("diff test",kv) for kv in diffApis])
def test_batchdiff(self, _, kv):
self.maxDiff = None
oldRes = requests.get(kv["oldapi"])
newRes = requests.get(kv["newapi"])
self.assertDictEqual(oldRes.json(), newRes.json())


if __name__ == '__main__':
unittest.main(verbosity=2)

Node.js示例

var assert = require('chai').assert;
var request = require('request');

describe("Node.js版本批量接口diff", function () {
this.timeout(0);
[
// diff apis
{
oldapi: "https://mtj.baidu.com/data/mobile/trend?dimension=brand&platformId=0&rankId=16&stattime=1527782400",
newapi: "https://mtj.baidu.com/data/mobile/trend?dimension=brand&platformId=0&rankId=16&stattime=1527782400"
},
{
oldapi: "https://mtj.baidu.com/data/mobile/trend?dimension=brand&platformId=0&rankId=16&stattime=1527782400",
newapi: "https://mtj.baidu.com/data/mobile/trend?dimension=brand&platformId=0&rankId=4&stattime=1527782400"
},
{
oldapi: "https://mtj.baidu.com/data/mobile/trend?dimension=brand&platformId=0&rankId=16&stattime=1527782400",
newapi: "https://mtj.baidu.com/data/mobile/trend?dimension=brand&platformId=0&rankId=23&stattime=1527782400"
},
{
oldapi: "https://mtj.baidu.com/data/mobile/trend?dimension=brand&platformId=0&rankId=16&stattime=1527782400",
newapi: "https://mtj.baidu.com/data/mobile/trend?dimension=brand&platformId=0&rankId=12&stattime=1527782400"
},
{
oldapi: "https://mtj.baidu.com/data/mobile/trend?dimension=brand&platformId=0&rankId=16&stattime=1527782400",
newapi: "https://mtj.baidu.com/data/mobile/trend?dimension=brand&platformId=0&rankId=2&stattime=1527782400"
},
{
oldapi: "https://mtj.baidu.com/data/mobile/trend?dimension=brand&platformId=0&rankId=16&stattime=1527782400",
newapi: "https://mtj.baidu.com/data/mobile/trend?dimension=brand&platformId=0&rankId=15&stattime=1527782400"
},
{
oldapi: "http://wis.qq.com/weather/common?callback=jQuery111105922904533110209_1533657946519&weather_type=observe%7Cforecast_24h%7Cair&source=pc&province=%E5%8C%97%E4%BA%AC%E5%B8%82&city=%E5%8C%97%E4%BA%AC%E5%B8%82&county=&_=1533657946520",
newapi: "http://wis.qq.com/weather/common?callback=jQuery111105922904533110209_1533657946519&weather_type=observe%7Cforecast_24h%7Cair&source=pc&province=%E5%8C%97%E4%BA%AC%E5%B8%82&city=%E5%8C%97%E4%BA%AC%E5%B8%82&county=&_=1533657946520"
},
{
oldapi: "http://wis.qq.com/weather/common?source=pc&weather_type=observe|alarm&province=%E5%8C%97%E4%BA%AC%E5%B8%82&city=%E5%8C%97%E4%BA%AC%E5%B8%82&county=&callback=jQuery112008061541882788377_1533657948645&_=1533657948647",
newapi: "http://wis.qq.com/weather/common?source=pc&weather_type=observe|alarm&province=%E5%8C%97%E4%BA%AC%E5%B8%82&city=%E5%8C%97%E4%BA%AC%E5%B8%82&county=&callback=jQuery112008061541882788377_1533657948645&_=1533657948647"
}
].forEach(function (kv) {
it(kv.newapi + " vs " + kv.oldapi, function (done) {
request.get(kv.newapi, {
json: true
},
function (err, res, body) {
request.get(kv.oldapi, {
json: true
},
function (oldapiError, oldapiRes, oldapiBody) {
assert.deepEqual(body, oldapiBody);
done();
}
);
}
);
});
});
});

备注

  1. pytest、pip如何使用请自行学习,本文不做介绍;
  2. mocha、npm如何使用请自行学习,本文不做介绍;
如果觉得我的文章对您有用,请随意打赏。您的支持将鼓励我继续创作!
共收到 10 条回复 时间 点赞

java的版本没有去实现,感兴趣的小伙伴自行github搜索json对比的开源项目

3Floor has been deleted

不错,楼主应该说的是主要的对比方法,但具体的执行方法有机会可以在补充一下哈
2点
1、不同版本的接口做diff测试,基本上至少要2套或以上测试环境,设1套为旧接口生产环境1套新接口diff环境,你这里是什么方式将流量从旧接口服务器复制到新接口
2、新接口环境的测试数据如何管理,还是说在diff环境做数据挡板,还有怎么保障流量到新接口走过的子调用和旧接口子调用的差异,数据比对相同不一定就是diff的真正结果

terrychow 回复

不同版本测试接口diff可以尝试下述思路:
思路1、部署两套测试环境,可以让oldJson等于旧版本接口返回值,newJson等于新接口返回值,然后进行对比;
思路2、如果只有一套测试环境,可以先使用CI工具部署旧版本源代码构建的应用,遍历需要diff的接口,存储各个接口返回值为json(存库也可)。接下来部署新版本源代码构建的应用,遍历所有接口,对每一个接口返回值进行对比。
思路3、如果允许使用线上发布的版本接口返回值(通常情况下面向公众的App、web应用均可使用线上版本)进行对比,思路与思路1相似。

terrychow 回复

第二点问题回复如下,仅供参考。
1、测试数据如何管理?
这个可以使用json文件、Excel、csv、数据库均可,看个人喜好,怎么快就怎么用。
2、如何保证新接口子调用是否一致?
方法1、让开发在调用过程中打印日志,通过检查日志来确认调用链路是否一致。
方法2、工作过程中如果有代码查看权限,直接确认代码变动可能更快;
欢迎其他小伙伴补充

推荐大家使用mocha搞定diff测试
1、高效,例如v1、v2两套接口,url中除了版本号不一样外,其他参数相同,稍微组织一下代码,一个forEach便可搞定成百上千的接口diff测试。

2、测试报告漂亮

3、Google、Microsoft、Facebook都在用,思考一下为什么世界顶级大公司都在用mocha?
问题抛给各位了

此外,文中mocha进行diff测试过程中参数化使用的是数组,当然可以选择json文件、csv文件、Excel文件、数据库数据进行参数。

提几个建议吧,你的代码实现:

  1. 写好注释,让代码可读性更强。
  2. 代码良好的设计,比如要穿插Post等,Http请求头不同,Https协议等等,代码如何更优雅。
  3. 接口返回值比对是在接口成功返回之后做的事,而接口的成功返回需要各种各样的断言,要加上。
qianxing #12 · August 13, 2018 作者
zyanycall 回复

收到你的建议,周末有时间再去改了

Author only
需要 Sign In 后方可回复, 如果你还没有账号请点击这里 Sign Up