通用技术 python + paramiko 实现自动部署替换文件

匿名 · 2017年06月16日 · 最后由 阳光下的草 回复于 2017年06月16日 · 2207 次阅读

背景

每次版本上线后,陆续会有现网反馈的 bug 需要修复,这个时候研发会给一些 web.zip 形式的替换文件进行测试、上线修复 bug。而每次在测试环境部署时,总是有人不进行备份,直接 cp 过去覆盖,后续如果有问题回滚比较麻烦,同时需要敲一堆命令,较为繁琐,为了方便操作,遂写了一个简单的脚本,实现自动上传、备份、部署的操作。

操作步骤

1.从 Jira 下载 web.zip 压缩包至本地并解压(个人是下载在 D:\file)
2.运行代码,如果需要重启服务器,输入:1 ,不需要重启则输入:0
3.喝杯水休息,等待服务器重启成功😀 😀 😀

主要代码展示

首先是一些下面需要用的方法,连接服务器、上传文件、执行 linux 命令、关闭服务器连接。
这些主要是 paramiko 这个库的一些操作,作为测试人员这一块可能接触的比较少,一般运维的工程师可能了解更多一点,我也只是懂一点皮毛,没有深入研究

def trans_connect(host,username,password):
    try:
        trans = paramiko.Transport((host,22))
        trans.connect(username=username,password=password)
    except Exception,e:
        print e
    return trans

def trans_web(trans,remotepath,localpath):
    sftp = paramiko.SFTPClient.from_transport(trans)
    sftp.put(localpath,remotepath)
    trans.close()

def ssh_connect(host,username,password):
    try:
        ssh_client = paramiko.SSHClient()
        ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
        ssh_client.connect(host,22,username,password)
    except Exception ,e:
        print e
    return ssh_client

def ssh_exec_cmd(ssh_client,cmd,arg):
    return ssh_client.exec_command(cmd,arg)

def ssh_close(ssh_client):
    ssh_client.close()

下面便是主要的脚本:
其中打印文件夹中的所有文件,不包括文件夹这几行代码,我借鉴百度上的一个 ideal,说实话我还没太看懂这段,蜜汁尴尬😰
为什么在执行命令的时候,要加上 arg 呢?
一开始我是默认 get_pty=True 写死的,后来发现在重启服务的时候 service tomcat_iorder_appsvr restart ,服务停掉就起不来了
后来找了不少资料,试着将 get_pty=False 重启了一遍服务,发现就好了,至于原因我也是没太了解,欢迎懂得兄弟帮我解答一下

def main():
    host = '172.31.x.xxx'                           
    username = 'root'
    password = 'xxxxxxxx123'
    remotepath = '/opt/web/web.zip'
    localpath = r'D:/file/web.zip'                  -----存放web.zip地址
    #删除并创建新的web文件夹
    print u'开始删除opt下web,并创建新web\n'
    cmd = 'cd /opt;rm -rf web;mkdir web;ls'
    arg = 'get_pty=True'
    ssh_client=ssh_connect(host,username,password)
    stdin, stdout, stderr = ssh_exec_cmd(ssh_client,cmd,arg)
    for line in stdout:
        print line.strip('\n')
    ssh_close(ssh_client)

    #上传文件
    print u'开始上传文件\n'
    trans = trans_connect(host,username=username,password=password)
    trans_web(trans,remotepath=remotepath,localpath=localpath)

    #解压并删除压缩包
    print u'解压并删除压缩包\n'
    cmd = 'cd /opt/web;ls;unzip web.zip;rm -rf web.zip;ls'
    ssh_client=ssh_connect(host,username,password)
    stdin, stdout, stderr = ssh_exec_cmd(ssh_client,cmd,arg)
    for line in stdout:
        print line.strip('\n')
    ssh_close(ssh_client)

    #备份需要替换的文件
    print u'开始备份需要替换的文件'
    path=r'D:\\file\\web'
    fns=[os.path.join(root,fn) for root,dirs,files in os.walk(path) for fn in files]
    for f in fns:
        oldfile = '/home/iorder_appsvr/iorder_appsvr'+ str(f[9:].replace('\\','/'))
        newfile = oldfile +  str(time.strftime("%y%m%d")) + 'bak'
        cmd = "mv %s  %s"%(oldfile,newfile)
        ssh_client=ssh_connect(host,username,password)
        stdin, stdout, stderr = ssh_exec_cmd(ssh_client,cmd,arg)
        for line in stdout:
            print line.strip('\n')
        ssh_close(ssh_client)
        time.sleep(1)
    print(u'总计:'+ str(len(fns)) +u'个   备份完成')

    #复制文件至appsvr
    ssh_client=ssh_connect(host,username,password)
    cmd = '\cp -Rf /opt/web/   /home/iorder_appsvr/iorder_appsvr/'
    stdin, stdout, stderr = ssh_exec_cmd(ssh_client,cmd,arg)
    for line in stdout:
        print line.strip('\n')
    time.sleep(2)

    #重启服务
    print u"是否需要重启服务器,是请输入1,否请输入0"
    result = raw_input(u'是否需要重启服务器:')
    result = int(result)
    if result == 1:
        cmd = 'service  tomcat_iorder_appsvr  restart'
        arg = 'get_pty=False'
        ssh_client=ssh_connect(host,username,password)
        stdin, stdout, stderr = ssh_exec_cmd(ssh_client,cmd,arg)
        for line in stdout:
            print line.strip('\n')
        time.sleep(2)
        ssh_close(ssh_client)
    else:
        ssh_close(ssh_client)

if __name__ == "__main__":
    main()

That's all

共收到 2 条回复 时间 点赞

统一 print 写法,要么都函数式要么都作为函数调用 (推荐后者)
用 except Exception as e 替代逗号的写法
不要捕获"Exception",你应该要知道要调用的函数会抛出什么异常
ssh_client 尝试使用 with,而不是手动指定何时关
尝试使用 format 来格式化字符串而不是%
既然函数内尝试捕获了异常,但依然像正常状态一样返回了一个不可用的实例,建议要么不要捕获异常,要么捕获之后加打印,然后把异常抛出交给上一级处理
感兴趣的话试用看看 argparse 来原做某些按需的处理,很多时候脚本不要做交互式的才更好用

顺便练练英语吧,如果你想知道 pty 参数问题的原因

https://github.com/fabric/fabric/issues/395

pty/tty = false

The bash shell launched connects to the stdout/stderr/stdin of the started process and is kept running until there is nothing attached to the sockets and it's children have exited. A good deamon process will ensure it doesn't wait for it's children to exit, fork a child process and then exit. When in this mode no SIGHUP will be sent to the child process by SSH. I believe this will work correctly for most scripts executing a process that handles deamonizing itself and doesn't need to be backgrounded. Where init scripts use '&' to background a process then it's likely that the main problem will be whether the backgrounded process ever attempts to read from stdin since that will trigger a SIGHUP if the session has been terminated.

pty/tty = true*

If the init script backgrounds the process started, the parent BASH shell will return an exit code to the SSH connection, which will in turn look to exit immediately since it isn't waiting on a child process to terminate and isn't blocked on stdout/stderr/stdin. This will cause a SIGHUP to be sent to the parent bash shell process group, which since job control is disabled in non-interactive mode in bash, will include the child processes just launched. Where a daemon process explicitly starts a new process session when forking or in the forked process then it or it's children won't receive the SIGHUP from the BASH parent process exiting. Note this is different from suspended jobs which will see a SIGTERM.

I suspect the problems around this only working sometimes has to do with a slight race condition. If you look at the standard approach to deamonizing - http://www.itp.uzh.ch/~dpotter/howto/daemonize, you'll see that in the code the new session is created by the forked process which may not be run before the parent exits, thus resulting the random sucess/failure behaviour mentioned above. A sleep statement will allow enough time for the forked process to have created a new session, which is why it works for some cases.

pty/tty = true and job control is explicitly enabled in bash

SSH won't connect to the stdout/stderr/stdin of the bash shell or any launched child processes, which will mean it will exit as soon as the parent bash shell started finished executing the requested commands. In this case, with job control explicitly enabled, any processes launched by the bash shell with '&' to background them will be placed into a separate session immediately and will not receive the SIGHUP signal when the the parent process to the BASH session exits (SSH connection in this case).

如果看不明白的话我可以简单解释下

匿名 在 基于 Python3+PyQt5 的自动替换部署工具介绍 中提及了此贴 10月24日 16:22
匿名 在 再见 2017,期待 2018 中提及了此贴 01月02日 19:33
需要 登录 后方可回复, 如果你还没有账号请点击这里 注册