该文原创为新潮质量保障技术团队中的 “上进的中年软件测试从业者”,用于技术交流分享

昨天下午一个小时的经历胜过过去一年的收获,论说话的艺术,大写的一个服。身边一直都有这样的大佬,不知为何今年的耳朵格外的好用。成长的路上不分年龄,养活一团春意思,撑起两根穷骨头

前言

运维部门的安全加固,让团队的小伙伴领到了性能测试重构的大活。简单一点说就是通过跳板机进行 ssh 交互的方式替换为更合理的 Jenkins slave 远程调度方案。最早的时候其实就应该采用这种一劳永逸的方案,现实又一次不留情面的扇了个大巴掌过来。采用 Jenkins slave 方案有如下的优势:

实现

Jenkins 封装

开源的 python-jenkins 安装后,就可以进行常规的 Jenkins 操作了。

以构建 job 为例,我们不难发现框架的底层还是调用的 requests 的 post 请求方式:

python 
     def build_job(self, name, parameters=None, token=None):
     '''Trigger build job.

     This method returns a queue item number that you can pass to
     :meth:`Jenkins.get_queue_item`. Note that this queue number is only
     valid for about five minutes after the job completes, so you should
     get/poll the queue information as soon as possible to determine the
     job's URL.

     :param name: name of job
     :param parameters: parameters for job, or ``None``, ``dict``
     :param token: Jenkins API token
     :returns: ``int`` queue item
     '''
     response = self.jenkins_request(requests.Request(
         'POST', self.build_job_url(name, parameters, token)))

另外开源的 jenkins-python 库同样给我们提供了更多了解 Jenkins 的机会(如果想对 Jenkins 进行自动化,这部分信息非常有用):

INFO = 'api/json'
PLUGIN_INFO = 'pluginManager/api/json?depth=%(depth)s'
CRUMB_URL = 'crumbIssuer/api/json'
WHOAMI_URL = 'me/api/json?depth=%(depth)s'
JOBS_QUERY = '?tree=%s'
JOBS_QUERY_TREE = 'jobs[url,color,name,%s]'
JOB_INFO = '%(folder_url)sjob/%(short_name)s/api/json?depth=%(depth)s'
JOB_NAME = '%(folder_url)sjob/%(short_name)s/api/json?tree=name'
ALL_BUILDS = '%(folder_url)sjob/%(short_name)s/api/json?tree=allBuilds[number,url]'
Q_INFO = 'queue/api/json?depth=0'
Q_ITEM = 'queue/item/%(number)d/api/json?depth=%(depth)s'
CANCEL_QUEUE = 'queue/cancelItem?id=%(id)s'
CREATE_JOB = '%(folder_url)screateItem?name=%(short_name)s'  # also post config.xml
CONFIG_JOB = '%(folder_url)sjob/%(short_name)s/config.xml'
DELETE_JOB = '%(folder_url)sjob/%(short_name)s/doDelete'
ENABLE_JOB = '%(folder_url)sjob/%(short_name)s/enable'
DISABLE_JOB = '%(folder_url)sjob/%(short_name)s/disable'
SET_JOB_BUILD_NUMBER = '%(folder_url)sjob/%(short_name)s/nextbuildnumber/submit'
COPY_JOB = '%(from_folder_url)screateItem?name=%(to_short_name)s&mode=copy&from=%(from_short_name)s'
RENAME_JOB = '%(from_folder_url)sjob/%(from_short_name)s/doRename?newName=%(to_short_name)s'
BUILD_JOB = '%(folder_url)sjob/%(short_name)s/build'
STOP_BUILD = '%(folder_url)sjob/%(short_name)s/%(number)s/stop'
BUILD_WITH_PARAMS_JOB = '%(folder_url)sjob/%(short_name)s/buildWithParameters'
BUILD_INFO = '%(folder_url)sjob/%(short_name)s/%(number)d/api/json?depth=%(depth)s'
BUILD_CONSOLE_OUTPUT = '%(folder_url)sjob/%(short_name)s/%(number)d/consoleText'
BUILD_ENV_VARS = '%(folder_url)sjob/%(short_name)s/%(number)d/injectedEnvVars/api/json' + \
    '?depth=%(depth)s'
BUILD_TEST_REPORT = '%(folder_url)sjob/%(short_name)s/%(number)d/testReport/api/json' + \
    '?depth=%(depth)s'
DELETE_BUILD = '%(folder_url)sjob/%(short_name)s/%(number)s/doDelete'
WIPEOUT_JOB_WORKSPACE = '%(folder_url)sjob/%(short_name)s/doWipeOutWorkspace'
NODE_LIST = 'computer/api/json?depth=%(depth)s'
CREATE_NODE = 'computer/doCreateItem'
DELETE_NODE = 'computer/%(name)s/doDelete'
NODE_INFO = 'computer/%(name)s/api/json?depth=%(depth)s'
NODE_TYPE = 'hudson.slaves.DumbSlave$DescriptorImpl'
TOGGLE_OFFLINE = 'computer/%(name)s/toggleOffline?offlineMessage=%(msg)s'
CONFIG_NODE = 'computer/%(name)s/config.xml'
VIEW_NAME = '%(folder_url)sview/%(short_name)s/api/json?tree=name'
VIEW_JOBS = 'view/%(name)s/api/json?tree=jobs[url,color,name]'
CREATE_VIEW = '%(folder_url)screateView?name=%(short_name)s'
CONFIG_VIEW = '%(folder_url)sview/%(short_name)s/config.xml'
DELETE_VIEW = '%(folder_url)sview/%(short_name)s/doDelete'
SCRIPT_TEXT = 'scriptText'
NODE_SCRIPT_TEXT = 'computer/%(node)s/scriptText'
PROMOTION_NAME = '%(folder_url)sjob/%(short_name)s/promotion/process/%(name)s/api/json?tree=name'
PROMOTION_INFO = '%(folder_url)sjob/%(short_name)s/promotion/api/json?depth=%(depth)s'
DELETE_PROMOTION = '%(folder_url)sjob/%(short_name)s/promotion/process/%(name)s/doDelete'
CREATE_PROMOTION = '%(folder_url)sjob/%(short_name)s/promotion/createProcess?name=%(name)s'
CONFIG_PROMOTION = '%(folder_url)sjob/%(short_name)s/promotion/process/%(name)s/config.xml'
LIST_CREDENTIALS = '%(folder_url)sjob/%(short_name)s/credentials/store/folder/' \
                    'domain/%(domain_name)s/api/json?tree=credentials[id]'
CREATE_CREDENTIAL = '%(folder_url)sjob/%(short_name)s/credentials/store/folder/' \
                    'domain/%(domain_name)s/createCredentials'
CONFIG_CREDENTIAL = '%(folder_url)sjob/%(short_name)s/credentials/store/folder/' \
                    'domain/%(domain_name)s/credential/%(name)s/config.xml'
CREDENTIAL_INFO = '%(folder_url)sjob/%(short_name)s/credentials/store/folder/' \
                    'domain/%(domain_name)s/credential/%(name)s/api/json?depth=0'
QUIET_DOWN = 'quietDown'

远程调用

result = f.jenkins_build_job(job_name=job,
                             params={"jmxFile": open("jmxFile", "rb"), "shFile": open("shFile", "rb")},
                             wait_for_finish=True, duration=5)
print(result)

问题来了,我的参数哪里去了????

尝试了各种办法,变换各种参数模式,甚至抓了包,尝试了切换回 python2.7 用 MultipartParam 来构造 mutipart/form-data 的数据还是无果。

最后还是从国外网站查到了

实际上这个时候团队成员已经通过 requests 的 post 解决了这个问题,但是我还是偏执的想通过已经成型的框架去解决这个问题,避免更多的二次封装,事实证明偶尔的偏执是有好处的,就好比这次。

再次封装(jenkinsapi 库)

def build_job_with_file_parameter(self, job_name, params=None, files=None, wait_until_finish=False):
        """
        构建文件类型参数的job
        :param job_name: job名称
        :param params: 参数
        :param files: 文件参数如:{"jmxFile": open("jmxFile", "rb"), "shFile": open("shFile", "rb")}
        :param wait_until_finish: 是否等待完成
        :return: 构建状态
        """
        return Job(url=self.url + "job/%s/" % job_name, name=job_name, jenkins_obj=self.jenkinsapi_server).invoke(
            block=wait_until_finish,
            build_params=params,
            files=files)

调优
之前的经验告诉我,Jenkins 的性能随着使用的多样性,尤其 job 长时间执行会出现卡死的情况,所以为了通用和非阻塞,这个时候必须要引入超时机制

处理超时
可能存在的方案如下:

网上有一篇文章对 threading 的 join 非常到位(参见:https://blog.csdn.net/zhiyuan_2007/article/details/48807761):

结语

事实又一次提醒我们,过去的那些政治领袖每天写日记是有原因的。就如我今天写的这篇,应该是我写测试平台以来举证最多,过程最具体的一次。


↙↙↙阅读原文可查看相关链接,并与作者交流