在近期工作规划中，分布式压测框架提上日程，目前FunTester已经具备了一些分布式压测中用到的功能。
例如在执行用例端：利用反射根据方法名执行方法的使用示例、命令行如何执行 jar 包里面的方法；
或者在用例编写端：如何统一接口测试的功能、自动化和性能测试用例、如何在 Linux 命令行界面愉快进行性能测试；
亦或前段时间探索的Groovy反射执行问题：反射执行 Groovy 类方法 NoSuchMethodException 解答、Groovy 反射 invokeMethod 传参实践。
目前看已经有了几种粗略的性能测试用例方案，有一些已经进行了实践，有一些已经被我放弃了。分享出来，算是个梳理。
这里的测试用例方案分为两类：用例传递和用例执行。目前我的想法还是通过HTTP协议接口完成用例的传递和执行中控制。采用定时任务或者脚本轮询的方式进行执行的控制。目前来看肯定是一个Springboot项目了，这都是后话了。
下面分享第一种设想：
这种测试场景应该说非常少了，基于单个或者多个固定的HttpRequestBase对象的分布式压测方案，其实实现起来有点大材小用了。简单的请求，没有参数化规则，没有上下游接口调用，没有前置后置处理，缺乏链路功能支持。说了这么多缺点，下面分享基于HttpRequestBase对象的优点：实现简单，用例传递非常好做。执行起来也直接可以使用框架提供的能力。兼容性好，可以直接从功能用例中提取部分用例然后执行，达到用例多用的目的。
这里需要区分用例来源。一般来讲，编写单个用例肯定绕不开一张图：
总体分成三部分：请求行、请求头和请求体。依照之前分享过的案例，将一个HttpRequestBase对象拆成三分部。例如我们获取一个请求的方式如下：
public static void main(String[] args) throws UnsupportedEncodingException {
HttpPost httpPost = getHttpPost("http://localhost:12345/test/qps");
httpPost.addHeader(getHeader("token", "324u2u09uweidlxnvldfsad"));
httpPost.setEntity(new StringEntity(getJson("name=FunTester").toString(), DEFAULT_CHARSET));
}
不过这样不利于HttpRequestBase对象在HTTP接口中传递，毕竟没有直接用序列化和反序列化的方法。所以我自己写了一个中间对象。
最开始的想法用fastJSON实现：
public static void main(String[] args) {
HttpPost httpPost = getHttpPost("http://localhost:12345/test/qps");
httpPost.addHeader(getHeader("token", "324u2u09uweidlxnvldfsad"));
httpPost.setEntity(new StringEntity(getJson("name=FunTester").toString(), DEFAULT_CHARSET));
httpPost.addHeader(HttpClientConstant.ContentType_JSON);
JSONObject httpResponse = getHttpResponse(httpPost);
output(httpResponse);
String s = JSON.toJSONString(httpPost);
HttpPost httpPost1 = JSON.parseObject(s, httpPost.getClass());
JSONObject httpResponse1 = getHttpResponse(httpPost1);
output(httpResponse1);
}
测试结果如下：
INFO-> 当前用户：oker，工作目录：/Users/oker/IdeaProjects/funtester/,系统编码格式:UTF-8,系统Mac OS X版本:10.16
WARN-> 响应体非json格式，已经自动转换成json格式！
INFO-> 请求uri：http://localhost:12345/test/qps , 耗时：236 ms , HTTPcode: 200
INFO->
~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~ JSON ~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~
＞ {
＞ ① . "code":-2,
＞ ① . "FunTester":200,
＞ ① . "content":"FunTester"
＞ }
~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~ JSON ~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~
WARN-> 获取post请求参数时异常！
WARN-> 获取请求相应失败！请求内容:{requestType='POST', host='', apiName='', uri='http://localhost:12345/test/qps', headers={"Connection":"keep-alive"}, args={}, params={}, json={}, response={}}
java.lang.UnsupportedOperationException: public abstract int org.apache.http.params.HttpParams.getIntParameter(java.lang.String,int)
at com.alibaba.fastjson.JSONObject.invoke(JSONObject.java:485) ~[fastjson-1.2.62.jar:?]
at com.sun.proxy.$Proxy22.getIntParameter(Unknown Source) ~[?:?]
at org.apache.http.client.params.HttpClientParamConfig.getRequestConfig(HttpClientParamConfig.java:59) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:177) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) ~[httpclient-4.5.6.jar:4.5.6]
at com.funtester.httpclient.FunLibrary.getHttpResponse(FunLibrary.java:352) [classes/:?]
at com.funtest.javatest.FF.main(FF.java:21) [classes/:?]
INFO-> json 对象是空的！
Process finished with exit code 0
请求发生错误，其实这里面POST请求实体拷贝会失败。所以这个方法行不通了，只能换一个自己实现的。
public static void main(String[] args) {
HttpPost httpPost = getHttpPost("http://localhost:12345/post");
httpPost.addHeader(getHeader("token", "324u2u09uweidlxnvldfsad"));
httpPost.setEntity(new StringEntity(getJson("name=FunTester").toString(), DEFAULT_CHARSET));
httpPost.addHeader(HttpClientConstant.ContentType_JSON);
JSONObject httpResponse = getHttpResponse(httpPost);
output(httpResponse);
FunRequest funRequest = FunRequest.initFromRequest(httpPost);
HttpRequestBase request = funRequest.getRequest();
JSONObject httpResponse1 = getHttpResponse(request);
output(httpResponse1);
String text = JSON.toJSONString(funRequest);
HttpRequestBase httpRequestBase = FunRequest.initFromString(text).getRequest();
JSONObject httpResponse2 = getHttpResponse(httpRequestBase);
output(httpResponse2);
}
测试结果如下：
INFO-> 当前用户：oker，工作目录：/Users/oker/IdeaProjects/funtester/,系统编码格式:UTF-8,系统Mac OS X版本:10.16
WARN-> 响应体非json格式，已经自动转换成json格式！
INFO-> 请求uri：http://localhost:12345/post , 耗时：242 ms , HTTPcode: 200
INFO->
~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~ JSON ~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~
＞ {
＞ ① . "code":-2,
＞ ① . "FunTester":200,
＞ ① . "content":"FunTester"
＞ }
~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~ JSON ~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~
WARN-> 响应体非json格式，已经自动转换成json格式！
INFO-> 请求uri：http://localhost:12345/post , 耗时：1 ms , HTTPcode: 200
INFO->
~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~ JSON ~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~
＞ {
＞ ① . "code":-2,
＞ ① . "FunTester":200,
＞ ① . "content":"FunTester"
＞ }
~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~ JSON ~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~
WARN-> 响应体非json格式，已经自动转换成json格式！
INFO-> 请求uri：http://localhost:12345/post , 耗时：1 ms , HTTPcode: 200
WARN-> 响应体非json格式，已经自动转换成json格式！
INFO-> 请求uri：http://localhost:12345/post , 耗时：2 ms , HTTPcode: 200
INFO->
~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~ JSON ~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~
＞ {
＞ ① . "code":-2,
＞ ① . "FunTester":200,
＞ ① . "content":"FunTester"
＞ }
~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~ JSON ~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~~☢~
Process finished with exit code 0
用例很简单，就是一个个的HttpRequestBase对象，这里用com.funtester.httpclient.FunRequest类作为中转。
static FunRequest getCase001() {
HttpPost httpPost = getHttpPost("http://localhost:12345/post");
httpPost.addHeader(getHeader("token", "324u2u09uweidlxnvldfsad"));
httpPost.setEntity(new StringEntity(getJson("name=FunTester").toString(), DEFAULT_CHARSET));
httpPost.addHeader(HttpClientConstant.ContentType_JSON);
return FunRequest.initFromRequest(httpPost);
}
这样我们获取一个FunRequest对象即可。
这里分两种：一种是从代码里面创建FunRequest，用字符串信息保存。
static String getCase001() {
HttpPost httpPost = getHttpPost("http://localhost:12345/post");
httpPost.addHeader(getHeader("token", "324u2u09uweidlxnvldfsad"));
httpPost.setEntity(new StringEntity(getJson("name=FunTester").toString(), DEFAULT_CHARSET));
httpPost.addHeader(HttpClientConstant.ContentType_JSON);
FunRequest funRequest = FunRequest.initFromRequest(httpPost);
return JSON.toJSONString(funRequest);
}
第二种是原本就是字符串，或者数据库中一条记录，通过某种不可描述的方法转换成FunRequest对象。这里就分一下从字符串中读取FunRequest对象的方法，由于情况过于复杂，其他的情况我就不写了，各位有兴趣可以自己实现。
/**
* 从字符串中获取请求对象
* @param fun
* @return
*/
static FunRequest initFromString(String fun) {
def f = JSON.parseObject(fun)
RequestType requestType = RequestType.getInstance(f.requestType)
def request = new FunRequest(requestType)
request.host = f.host
request.path = f.path
request.uri = f.uri
request.args = f.args
request.json = f.json
request.params = f.params
f.headers.each {
request.addHeader(it.name,it.value)
}
request
}
这个相对接简单多了。用例一旦转换成字符串之后，就可以通过接口上传到master服务，或者由master服务分配给salve服务（暂时Springboot方案）去执行。
这里先写一个简单的POST接口上传用例的 Demo。
static boolean updateCase() {
HttpPost httpPost = getHttpPost("http://localhost:12345/updatecase");
httpPost.addHeader(getHeader("token", "324u2u09uweidlxnvldfsad"));
JSONObject json = getJson("name=FunTester");
json.put("FunTester", getCase002());
json.put("各种参数", FunTester);
httpPost.setEntity(new StringEntity(json.toString(), DEFAULT_CHARSET));
httpPost.addHeader(HttpClientConstant.ContentType_JSON);
JSONObject httpResponse = getHttpResponse(httpPost);
return isRight(httpResponse);
}
这里我目前是放弃Socket协议推送用例，而是让salve去master分配好的队列中取用例，当然这里的用例包含必要的运行信息，而不仅仅是一个FunRequest对象。
static JSONObject returnCase() {
JSONObject json = getJson("name=FunTester");
json.put("FunTester", getCase002());
json.put("各种参数", "FunTester");
return json;
}
salve拿到用例之后，先去解析响应，然后通过构建多线程任务对象或者list，然后交付给执行框架去完成用例的执行和数据展示和记录。
RequestThreadTimes模型做了 Demo，其他三种暂时不赘述了。相对简单，虽然可以通过组合多个性能用例来完成多个HttpRequestBase对象的性能压测，但是我非常不推荐这样，所以我在接下来的对象里面拒绝这种用例形式。
static void executeCase(CaseBase caseBase) {
FunRequest funRequest = caseBase.getFunRequest();
int thread = caseBase.getThread();
int times = caseBase.getTimes();
int runup = caseBase.getRunup();
String name = caseBase.getName();
RequestThreadTimes requestThreadTimes = new RequestThreadTimes(funRequest.getRequest(), times);
Concurrent concurrent = new Concurrent(requestThreadTimes, thread, name);
Constant.RUNUP_TIME = runup * 1.0;
concurrent.start();
}
这种用力的话，我采取的方案是caseBase对象中做一下区分，获取用例解析的时候解析成一个list，然后通过线程参数thread从头开始去list取FunRequest对象，构造多线程任务类RequestThreadTimes，完事儿之后交给执行框架执行。
static void executeCase(CaseBase caseBase) {
List<FunRequest> funRequests = caseBase.getFunRequests();
int thread = caseBase.getThread();
int times = caseBase.getTimes();
int runup = caseBase.getRunup();
String name = caseBase.getName();
List<ThreadBase> collect = range(thread).mapToObj(f -> new RequestThreadTimes(funRequests.get(f % funRequests.size()).getRequest(), times)).collect(Collectors.toList());
Concurrent concurrent = new Concurrent(collect, name);
Constant.RUNUP_TIME = runup * 1.0;
concurrent.start();
}
package com.funtester.httpclient
import com.alibaba.fastjson.JSON
import com.alibaba.fastjson.JSONObject
import com.funtester.base.bean.RequestInfo
import com.funtester.base.exception.RequestException
import com.funtester.config.HttpClientConstant
import com.funtester.config.RequestType
import com.funtester.frame.Save
import com.funtester.frame.SourceCode
import com.funtester.utils.Time
import org.apache.commons.lang3.StringUtils
import org.apache.http.Header
import org.apache.http.HttpEntity
import org.apache.http.client.methods.HttpPost
import org.apache.http.client.methods.HttpPut
import org.apache.http.client.methods.HttpRequestBase
import org.apache.http.client.methods.RequestBuilder
import org.apache.logging.log4j.LogManager
import org.apache.logging.log4j.Logger
/**
* 重写FunLibrary，使用面对对象思想,不用轻易使用set属性方法,可能存在BUG
*/
class FunRequest extends SourceCode implements Serializable, Cloneable {
private static final long serialVersionUID = -4153600036943378727L
private static Logger logger = LogManager.getLogger(FunRequest.class)
/**
* 请求类型，true为get，false为post
*/
RequestType requestType
/**
* 请求对象
*/
HttpRequestBase request
/**
* host地址
*/
String host = EMPTY
/**
* 接口地址
*/
String path = EMPTY
/**
* 请求地址,如果为空则由host和path拼接
*/
String uri = EMPTY
/**
* header集合
*/
List<Header> headers = new ArrayList<>()
/**
* get参数
*/
JSONObject args = new JSONObject()
/**
* post参数,表单
*/
JSONObject params = new JSONObject()
/**
* json参数,用于POST和put
*/
JSONObject json = new JSONObject()
/**
* 响应,若没有这个参数,从将funrequest对象转换成json对象时会自动调用getresponse方法
*/
JSONObject response = new JSONObject()
/**
* 构造方法
*
* @param requestType
*/
private FunRequest(RequestType requestType) {
this.requestType = requestType
}
/**
* 获取get对象
*
* @return
*/
static FunRequest isGet() {
new FunRequest(RequestType.GET)
}
/**
* 获取post对象
*
* @return
*/
static FunRequest isPost() {
new FunRequest(RequestType.POST)
}
/**
* 获取put请求对象
* @return
*/
static FunRequest isPut() {
new FunRequest(RequestType.PUT)
}
/**
* 获取delete请求对象
* @return
*/
static FunRequest isDelete() {
new FunRequest(RequestType.DELETE)
}
/**
* 设置host
*
* @param host
* @return
*/
FunRequest setHost(String host) {
this.host = host
this
}
/**
* 设置接口地址
*
* @param path
* @return
*/
FunRequest setpath(String path) {
this.path = path
this
}
/**
* 设置uri
*
* @param uri
* @return
*/
FunRequest setUri(String uri) {
this.uri = uri
this
}
/**
* 添加get参数
*
* @param key
* @param value
* @return
*/
FunRequest addArgs(Object key, Object value) {
args.put(key, value)
this
}
/**
* 添加post参数
*
* @param key
* @param value
* @return
*/
FunRequest addParam(Object key, Object value) {
params.put(key, value)
this
}
/**
* 添加json参数
*
* @param key
* @param value
* @return
*/
FunRequest addJson(Object key, Object value) {
json.put(key, value)
this
}
/**
* 添加header
*
* @param key
* @param value
* @return
*/
FunRequest addHeader(Object key, Object value) {
headers << FunLibrary.getHeader(key.toString(), value.toString())
this
}
/**
* 添加header
*
* @param header
* @return
*/
FunRequest addHeader(Header header) {
headers.add(header)
this
}
/**
* 批量添加header
*
* @param header
* @return
*/
FunRequest addHeader(List<Header> header) {
header.each {h -> headers << h}
this
}
/**
* 增加header中cookies
*
* @param cookies
* @return
*/
FunRequest addCookies(JSONObject cookies) {
headers << FunLibrary.getCookies(cookies)
this
}
FunRequest addHeaders(List<Header> headers) {
this.headers.addAll(headers)
this
}
FunRequest addHeaders(JSONObject headers) {
headers.each {x ->
this.headers.add(FunLibrary.getHeader(x.getKey().toString(), x.getValue().toString()))
}
this
}
FunRequest addArgs(JSONObject args) {
this.args.putAll(args)
this
}
FunRequest addParams(JSONObject params) {
this.params.putAll(params)
this
}
FunRequest addJson(JSONObject json) {
this.json.putAll(json)
this
}
/**
* 获取请求响应，兼容相关参数方法，不包括file
*
* @return
*/
JSONObject getResponse() {
response = response.isEmpty() ? FunLibrary.getHttpResponse(request == null ? getRequest() : request) : response
response
}
/**
* 获取请求对象
*
* @return
*/
HttpRequestBase getRequest() {
if (request != null) request
if (StringUtils.isEmpty(uri))
uri = host + path
switch (requestType) {
case RequestType.GET:
request = FunLibrary.getHttpGet(uri, args)
break
case RequestType.POST:
request = !params.isEmpty() ? FunLibrary.getHttpPost(uri + FunLibrary.changeJsonToArguments(args), params) : !json.isEmpty() ? FunLibrary.getHttpPost(uri + FunLibrary.changeJsonToArguments(args), json.toString()) : FunLibrary.getHttpPost(uri + FunLibrary.changeJsonToArguments(args))
break
case RequestType.PUT:
request = FunLibrary.getHttpPut(uri, json)
break
case RequestType.DELETE:
request = FunLibrary.getHttpDelete(uri)
break
case RequestType.PATCH:
request = FunLibrary.getHttpPatch(uri, params)
default:
break
}
for (Header it : headers) {
if (it.getName() != HttpClientConstant.ContentType_JSON.getName()) request.addHeader(it)
}
logger.debug("请求信息：{}", new RequestInfo(this.request).toString())
request
}
FunRequest setHeaders(List<Header> headers) {
this.headers = headers
this
}
FunRequest setArgs(JSONObject args) {
this.args = args
this
}
FunRequest setParams(JSONObject params) {
this.params = params
this
}
FunRequest setJson(JSONObject json) {
this.json = json
this
}
@Override
FunRequest clone() {
initFromRequest(this.getRequest())
}
@Override
String toString() {
return "{" +
"requestType='" + requestType.getName() + '\'' +
", host='" + host + '\'' +
", path='" + path + '\'' +
", uri='" + uri + '\'' +
", headers=" + FunLibrary.header2Json(headers).toString() +
", args=" + args.toString() +
", params=" + params.toString() +
", json=" + json.toString() +
", response=" + response.toString() +
'}'
}
/**
* 将请求对象转成curl命令行
* @return
*/
String toCurl() {
StringBuffer curl = new StringBuffer("curl -w HTTPcode%{http_code}:代理返回code%{http_connect}:数据类型%{content_type}:DNS解析时间%{time_namelookup}:%{time_redirect}:连接建立完成时间%{time_pretransfer}:连接时间%{time_connect}:开始传输时间%{time_starttransfer}:总时间%{time_total}:下载速度%{speed_download}:speed_upload%{speed_upload} ")
curl << " -X ${requestType.getName()} "
headers.each {
curl << " -H '${it.getName()}:${it.getValue().replace(SPACE_1, EMPTY)}'"
}
switch (requestType) {
case RequestType.GET:
args.each {
curl << " -d '${it.key}=${it.value}'"
}
break
case RequestType.POST:
if (!params.isEmpty()) {
curl << " -H Content-Type:application/x-www-form-urlencoded"
params.each {
curl << " -F '${it.key}=${it.value}'"
}
}
if (!json.isEmpty()) {
curl << " -H \"Content-Type:application/json\"" //此处多余,防止从外部构建curl命令
json.each {
curl << " -d '${it.key}=${it.value}'"
}
}
break
default:
break
}
curl << " ${uri}"
// curl << " --compressed" //这里防止生成多个curl请求,批量生成有用
curl.toString()
}
/**
* 将请求对象转成curl命令行
* @param requestBase
* @return
*/
static String reqToCurl(HttpRequestBase requestBase) {
initFromRequest(requestBase).toCurl()
}
/**
* 从requestbase对象从初始化funrequest
* @param base
* @return
*/
static FunRequest initFromRequest(HttpRequestBase base) {
FunRequest request = null
String method = base.getMethod()
String uri = base.getURI().toString()
RequestType requestType = RequestType.getInstance(method)
List<Header> headers = Arrays.asList(base.getAllHeaders())
if (requestType == requestType.GET) {
request = isGet().setUri(uri).addHeaders(headers)
} else if (requestType == RequestType.POST) {
HttpPost post = (HttpPost) base
HttpEntity entity = post.getEntity()
if (entity == null) {
request = isPost().setUri(uri).addHeader(headers)
} else {
Header type = entity.getContentType()
String value = type == null ? EMPTY : type.getValue()
String content = FunLibrary.getContent(entity)
if (value.equalsIgnoreCase(HttpClientConstant.ContentType_TEXT.getValue()) || value.equalsIgnoreCase(HttpClientConstant.ContentType_JSON.getValue())) {
request = isPost().setUri(uri).addHeaders(headers).addJson(JSONObject.parseObject(content))
} else if (value.equalsIgnoreCase(HttpClientConstant.ContentType_FORM.getValue())) {
request = isPost().setUri(uri).addHeaders(headers).addParams(getJson(content.split("&")))
}
}
} else if (requestType == RequestType.PUT) {
HttpPut put = (HttpPut) base
String content = FunLibrary.getContent(put.getEntity())
request = isPut().setUri(uri).addHeaders(headers).setJson(JSONObject.parseObject(content))
} else if (requestType == RequestType.DELETE) {
request = isDelete().setUri(uri)
} else {
RequestException.fail("不支持的请求类型!")
}
return request
}
/**
* 从字符串中获取请求对象
* @param fun
* @return
*/
static FunRequest initFromString(String fun) {
def f = JSON.parseObject(fun)
RequestType requestType = RequestType.getInstance(f.requestType)
def request = new FunRequest(requestType)
request.host = f.host
request.path = f.path
request.uri = f.uri
request.args = f.args
request.json = f.json
request.params = f.params
f.headers.each {
request.addHeader(it.name,it.value)
}
request
}
static HttpRequestBase doCopy(HttpRequestBase base) {
(HttpRequestBase) RequestBuilder.copy(base).build()
}
/**
* 拷贝HttpRequestBase对象
* @param base
* @return
*/
static HttpRequestBase cloneRequest(HttpRequestBase base) {
initFromRequest(base).getRequest()
}
/**
* 保存请求和响应
* @param base
* @param response
*/
static void save(HttpRequestBase base, JSONObject response) {
FunRequest request = initFromRequest(base)
request.setResponse(response)
Save.info("/request/" + Time.getDate().substring(8) + SPACE_1 + request.getUri().replace(OR, CONNECTOR).replaceAll("https*:_+", EMPTY), request.toString())
}
}
