背景

接口测试过程中，当接口版本发生变更时，如何快速进行 diff 测试？本文使用 Node.js、Python、Java 三种语言来讲解如何快速的进行接口 diff 测试.

原理

通过对接口返回结果添加断言来判断不同版本接口返回值是否相同，结合测试框架动态生成测试用例（测试数据参数化）来批量对比接口返回值。

方法

  1. Node.js
    >> 断言使用 chai 的 assert.deepEqual 方法.

测试过程中常用命令：

# 创建node.js项目
npm init
# 安装依赖包
npm install request mocha chai mochawesome --save
# 执行测试用例，并指定报告为mochawesome，报告默认存放到当前项目 mochawesome-report/ 目录中
mocha test\batchdiff.test.js --reporter mochawesome
  1. Python
    >> 断言使用 unittest 的 assertDictEqual 方法.

测试过程中常用命令：

# 安装依赖包
pip install requests pytest pytest-html
# 执行测试用例，并生成html报告，报告指定存放到当前项目 static/ 目录中
pytest test\test_batchdiff.py --html=static\jsondiff.html
  1. Java
    >> 断言使用 junit 的 Assert.assertEquals 方法.

本地测试项目为 maven 项目，maven 依赖包如下： 

<dependency>
  <groupId>junit</groupId>
  <artifactId>junit</artifactId>
  <version>4.11</version>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>org.testng</groupId>
  <artifactId>testng</artifactId>
  <version>6.14.3</version>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>com.alibaba</groupId>
  <artifactId>fastjson</artifactId>
  <version>1.2.49</version>
</dependency>
<dependency>
  <groupId>com.github.kevinsawicki</groupId>
  <artifactId>http-request</artifactId>
  <version>6.0</version>
</dependency>
<dependency>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-report-plugin</artifactId>
  <version>2.22.0</version>
</dependency>

maven 命令： 

# 运行测试用例
mvn test
# 运行测试用例，生成html报告，报告默认存放到项目的 target/site 目录中
mvn  surefire-report:report

Node.js diff 测试示例

  1. 测试用例运行效果

  2. html 报告

  3. 源代码 

var assert = require('chai').assert;
var request = require('request');

describe("Node.js版本批量接口diff", function () {
  this.timeout(0);
  [
    // diff apis
    {
      "oldapi": "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/demo",
      "newapi": "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/demo"
    },
    {
      "oldapi": "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/hello",
      "newapi": "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/hello"
    },
    {
      "oldapi": "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/user",
      "newapi": "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/user"
    },
    {
      "oldapi": "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/1/list",
      "newapi": "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/1/list"
    },
    {
      "oldapi": "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/2/list",
      "newapi": "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/2/list"
    },
    {
      "oldapi": "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/3/list",
      "newapi": "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/3/list"
    },
  ].forEach(function (kv) {
    it(kv.newapi + " vs " + kv.oldapi, function (done) {
      request.get(kv.newapi, {
          json: true
        },
        function (err, res, body) {
          request.get(kv.oldapi, {
              json: true
            },
            function (oldapiError, oldapiRes, oldapiBody) {
              assert.deepEqual(body, oldapiBody);
              done();
            }
          );
        }
      );
    });
  });
});

Python diff 测试示例

  1. 测试用例运行效果

  2. html 报告

  3. 源代码 

# -*- coding:utf-8 -*-
import unittest
import requests
from parameterized import parameterized, param
diffApis = [
    # diff apis
    {
        "oldapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/demo",
        "newapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/demo"
    },
    {
        "oldapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/hello",
        "newapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/hello"
    },
    {
        "oldapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/user",
        "newapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/user"
    },
    {
        "oldapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/1/list",
        "newapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/1/list"
    },
    {
        "oldapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/2/list",
        "newapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/2/list"
    },
    {
        "oldapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/3/list",
        "newapi":"https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/3/list"
    },
]


class TestParameterized(unittest.TestCase):
    @parameterized.expand(
        [param("diff test",kv) for kv in diffApis])
    def test_batchdiff(self, _, kv):
        self.maxDiff = None
        oldRes = requests.get(kv["oldapi"])
        newRes = requests.get(kv["newapi"])
        self.assertDictEqual(oldRes.json(), newRes.json())


if __name__ == '__main__':
    unittest.main(verbosity=2)

Java diff 测试示例

  1. 测试用例运行效果

  1. html 报告

  2. 源代码 

package com.test.apitest;

import com.alibaba.fastjson.JSON;
import org.junit.Assert;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;
import com.github.kevinsawicki.http.HttpRequest;

public class BatchDiffTest {
    @DataProvider
    public Object[][] data() {
        Object[][] diffApis = new Object[][]{
                {
                        "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/demo",
                        "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/demo"
                },
                {
                        "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/hello",
                        "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/hello"
                },
                {
                        "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/user",
                        "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/user"
                },
                {
                        "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/1/list",
                        "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/1/list"
                },
                {
                        "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/2/list",
                        "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/2/list"
                },
                {
                        "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v1/3/list",
                        "https://easy-mock.com/mock/5b6b06f9a40bfb27425bbb6a/jsondiff/v2/3/list"
                },
        };
        return diffApis;
    }

    @Test(dataProvider = "data")
    public void test_apiDiff(String oldApiUrl, String newApiUrl) {
        String oldJsonString = HttpRequest.get(oldApiUrl)
                .accept("application/json")
                .body();
        String newJsonString = HttpRequest.get(newApiUrl)
                .accept("application/json")
                .body();
        Object oldJsonVo = JSON.parseObject(oldJsonString);
        Object newJsonVo = JSON.parseObject(newJsonString);
        Assert.assertEquals(oldJsonVo, newJsonVo);
    }
}

Node.js、Python、Java 对比效果分析

  1. 命令行日志显示的对比结果

    当接口返回结果不一致时：

    1) mocha+chai 断言在日志中显示效果最好，日志输出 json 字段级别不一致的地方；

    2) python+unittest 断言在日志中显示的是 python dict 对象，不太友好；

    3) java+junit 仅在日志中显示两个对比的对象不一致，需要借助 IDE 对比返回值字段级别不一致的地方；

  2. html 报告

    与命令行输出日志相似，当接口返回值不一致时：

    1) Node.js 体系的 mochawesome 报告最好看，最直观，可以看到 json 字段级别不一致的地方；

    2) Python 体系的 pytest-html 报告 diff 测试结果不友好，不能明显看出 json 字段级别不一致的地方；
    3) Java 体系的 Surefire Report 不能看出 json 不一致的地方；

  3. 中文支持

    Node.js、Java 语言输出日志、html 报告中中文字符正常显示为中文字符，python 输出日志、html 报告中中文字符显示为中文字符对应的 Unicode 编码

参考资料

各种测试框架都有官方文档，从官方文档中可以获取到参数化的知识，下述参考资料仅供参考.

  1. mocha
  2. 测试框架 Mocha 实例教程
  3. pytest
  4. pytest 官方帮助文档参数化
  5. pytest 文档 9-参数化 parametrize
  6. Junit 4
  7. testNg 参数化
  8. testng 参数化方法：Parameters 和 DataProvider

申明

本文发布于个人微信公众号 发现 Bug，欢迎关注。


↙↙↙阅读原文可查看相关链接，并与作者交流