自动化工具 谷歌开源模糊测试工具 ClusterFuzz 尝鲜记录 (进行中)
背景
模糊测试，是指用随机坏数据（也称做 fuzz）攻击一个程序，然后等着观察哪里遭到了破坏。（出自 模糊测试）。一直以来都有不少的模糊测试工具，但大多只集中在数据生成，执行和异常检测依赖人工，未有比较完整的方案。
早在八年前，google 内部就在建设和使用模糊测试的工具来测试其内部的应用，而在两年前， google 推出了 OSS-Fuzz 服务，用于给开源项目的进行免费的模糊测试服务，可自动在新版本代码提交后自动完成 测试->异常检测->issue 登记->老版本 issue 回归及自动关闭 的功能。背后使用的就是 ClusterFuzz 技术。流程图如下：
而在过年前，google 开源了 ClusterFuzz ，并解决了原有 ClusterFuzz 必须依赖 Google Cloud 提供的服务这个问题，提供了本地运行的解决方案。根据官方介绍，它具备如下功能：
- 高度可扩展，谷歌的内部实例运行在超过 25000 台机器上
- 准确的去副本化（Accurate deduplication）
- 问题跟踪器的全自动错误归档和关闭
- 最小化测试用例
- 通过二分法回归查找
- 提供分析 fuzzer 性能和崩溃率的统计信息（不支持本地部署）
- 易于使用的 Web 界面，用于管理和查看崩溃
- 支持引导模糊（例如 libFuzzer 和 AFL）和黑盒模糊测试
其大致执行流程如下：
当然，方案并不完美，如模糊数据统计、崩溃数据统计等功能由于依赖 google cloud 强大的数据处理能力，本地运行时是用不了的。
官方说的总是美好的，现实是否这么完美呢？曾有人说，实践是检验真理的唯一标准，为了更好地了解这个工具，当然就要本地跑个 demo 玩下啦。
本地搭建及运行
要获得 ClusterFuzz 的完整功能，需要连接Google Cloud Platform。但结合国情，我们更期望了解它纯本地运行能做到什么，因此这次尝鲜主要尝试纯本地运行。
注意：虽然运行可以脱离 Google Cloud Platform ，但部分安装时用到的工具需要到 google 站点下载，所以，你懂得。
以下步骤均是在 macOS 10.14 上进行。
环境搭建
1、下载源码
git clone https://github.com/google/clusterfuzz
cd clusterfuzz
2、安装 google cloud sdk
进入 https://cloud.google.com/sdk/ ，按照引导安装 sdk 并配置好环境变量（mac 下可以直接用解压后的 install.sh 脚本一键安装），确认命令行可调用 gcloud 命令
$ gcloud -v
Google Cloud SDK 226.0.0
bq 2.0.38
core 2018.11.16
gsutil 4.34
3、安装 python 和 go 运行环境。
特别注意：如果你使用的是 macOS 或者 Ubuntu、Debain，直接执行第 4 步即可，脚本里会自动安装 Python 和 go
python 要求 2.7.10 以上，但不能是 python 3。在 mac 上可以直接运行 brew install python@2 安装。
go 未要求版本，在 mac 上可以直接运行 brew install go 安装。我用的是 go1.11.5 darwin/amd64
4、安装其他依赖
针对
- Ubuntu (14.04, 16.04, 17.10, 18.04, 18.10)
- Debian 8 (jessie) or later
- Recent versions of macOS with homebrew (experimental)
几个系统，官方已经内置了安装依赖的脚本，直接运行即可：
local/install_deps.bash
执行完毕，会出现
Installation succeeded!
Please load virtualenv environment by running 'source ENV/bin/activate'.
的提示。
坑一，官方的脚本里第一行用了 -ex 参数，会导致运行脚本时如果有命令执行出错（如 brew install 时有些应用本地已经安装过，但非最新版本），直接退出程序。
可以通过 sed -i '' 's/bash -ex/bash -x/' local/install_deps* 命令直接去掉 -e 参数。已经给官方提了 issue
坑二，官方脚本里使用 python butler.py bootstrap 初始化环境时，会自动去 google 站点下载 chromedriver 相关的文件。
全局搜索了下源代码，只有跑单测的时候有用到 chromedriver ，所以可以直接注释掉这个函数：
diff --git a/src/local/butler/common.py b/src/local/butler/common.py
index 94b17b3..3e9de99 100644
--- a/src/local/butler/common.py
+++ b/src/local/butler/common.py
@@ -275,7 +275,7 @@ def install_dependencies(platform_name=None):
_remove_invalid_files()
execute('bower install --allow-root')
- _install_chromedriver()
+ #_install_chromedriver()
def symlink(src, target):
坑三，运行时会报错 Analysis of target '//local:create_gopath' failed; build aborted: no such package '@org_golang_google_api//iterator': failed to fetch org_golang_google_api: 2019/02/19 01:15:41 unrecognized import path "google.golang.org/api"
这是在运行 bazel 构建 go 环境的时候报错了，原因是 @org_golang_x_tools、@com_google_cloud_go、@org_golang_google_api 这几个第三方依赖网络原因获取不到。
尝试一：使用代理
因为 go 获取依赖有可能用 http ，也有可能用 git ，所以保险起见全部都配好代理：
export HTTP_PROXY=http://112.126.81.122:6$(date +%m%d)
export HTTPS_PROXY=${HTTP_PROXY}
git config --global https.proxy ${HTTP_PROXY}
git config --global http.proxy ${HTTP_PROXY}
可惜的是，配置完了还是不行，bazel 构建时提示 fatal: unable to access 'https://code.googlesource.com/google-api-go-client/': LibreSSL SSL_connect: SSL_ERROR_SYSCALL in connection to code.googlesource.com:443，此路不通。
尝试二：修改运行环境，改为在网络本身就没问题的地方运行
嗯，哪里有这样的环境呢？一个是自己买云主机，另一个就是考虑用 docker hub 提供的构建环境了。看了下后面的使用步骤，也没有需要在源码目录做操作的部分，就选择 docker 吧。
动手 fork 了官方仓库，开始了漫长的尝试：https://github.com/chenhengjie123/clusterfuzz
2.23 更新：docker 镜像已成功打包，基于 ubuntu 16.04 系统。镜像中已运行完毕本文中的第 1-4 步（除了坑 2 中的注释 chromedriver ），装好了所有依赖。镜像地址：https://hub.docker.com/r/chenhengjie123/clusterfuzz_local
可通过 docker run -it --name clusterfuzz --network host chenhengjie123/clusterfuzz_local 进入镜像运行环境，进入后续的步骤。clusterfuzz 的源代码存放在镜像的 /clusterfuzz 目录。
5、切换到 python 的 virtualenv
$ source ENV/bin/activate
校验是否一切就绪
$ python butler.py --help
python butler.py --help
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
usage: butler.py [-h]
{bootstrap,py_unittest,go_unittest,js_unittest,format,lint,package,deploy,run_server,run,run_bot,remote,clean_indexes,generate_datastore_models,create_config}
...
运行本地实例
本地实例包含 2 个部分，一个是管理各个执行机器人的服务端，另一个是执行机器人。
启动本地服务
首次运行，添加 --bootstrap 进行各个数据的初始化。同时个人推荐加上 --skip-install-deps 跳过依赖安装（前面步骤已经装过了，不需要重复安装）
$ python butler.py run_server --bootstrap --skip-install-deps
非首次运行，务必去掉 --bootstrap 参数。
坑四：启动时会到 https://www.googleapis.com/discovery/v1/apis/pubsub/v1/rest 获取一些信息，如果此时网络连不通，会报错
报错信息：
Created symlink: source: /clusterfuzz/local/storage/local_gcs, target /clusterfuzz/src/appengine/local_gcs.
Traceback (most recent call last):
File "butler.py", line 282, in <module>
main()
File "butler.py", line 256, in main
command.execute(args)
File "src/local/butler/run_server.py", line 162, in execute
test_utils.setup_pubsub(constants.TEST_APP_ID)
File "/clusterfuzz/src/python/tests/test_libs/test_utils.py", line 308, in setup_pubsub
_create_pubsub_topic(client, project, queue['name'])
File "/clusterfuzz/src/python/tests/test_libs/test_utils.py", line 284, in _create_pubsub_topic
if client.get_topic(full_name):
File "/clusterfuzz/src/python/google_cloud_utils/pubsub.py", line 192, in get_topic
request = self._api_client().projects().topics().get(topic=name)
File "/clusterfuzz/src/python/base/retry.py", line 88, in _wrapper
result = func(*args, **kwargs)
File "/clusterfuzz/src/python/google_cloud_utils/pubsub.py", line 89, in _api_client
discovery.DISCOVERY_URI.format(api='pubsub', apiVersion='v1'))
File "/clusterfuzz/src/third_party/httplib2/__init__.py", line 1694, in request
(response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
File "/clusterfuzz/src/third_party/httplib2/__init__.py", line 1434, in _request
(response, content) = self._conn_request(conn, request_uri, method, body, headers)
File "/clusterfuzz/src/third_party/httplib2/__init__.py", line 1390, in _conn_request
response = conn.getresponse()
File "/usr/lib/python2.7/httplib.py", line 1123, in getresponse
raise ResponseNotReady()
httplib.ResponseNotReady
解决猜想：看了下这个页面，实际上是获取 api 文档。理论上只要把这个 api 文档事先下载好并放到资源文件中，然后把这个从网络获取文档的步骤改为读取资源文件即可。晚些尝试下。
由于时间关系，暂时先想办法让网络能访问 google 先绕过。
启动到末尾，会出现如下日志：
| INFO 2019-02-23 06:25:34,648 api_server.py:265] Starting gRPC API server at: http://localhost:39957
| INFO 2019-02-23 06:25:34,877 dispatcher.py:256] Starting module "default" running at: http://localhost:9000
| INFO 2019-02-23 06:25:35,021 dispatcher.py:256] Starting module "cron-service" running at: http://localhost:9001
| INFO 2019-02-23 06:25:35,023 admin_server.py:150] Starting admin server at: http://localhost:9002
表明已经启动完毕。可以通过打开 http://localhost:9002/ 打开管理员界面。
坑五：内部监听地址都是 localhost ，意味着在 docker 容器内部时，即使用 -p 暴露了端口也访问不了
由 @lion-roadbike 的提供的解决方案：
替换启动命令中的地址即可
sed -i "s/--admin_port={admin_port}/--admin_port={admin_port} --admin_host=0.0.0.0 --host=0.0.0.0 --enable_host_checking=false /" /clusterfuzz/src/local/butler/run_server.py
sed -i "s/--env_var LOCAL_GCS_SERVER_HOST={local_gcs_server_host}/--env_var LOCAL_GCS_SERVER_HOST=http:\/\/172.17.0.2:9008/" /clusterfuzz/src/local/butler/run_server.py
sed -i "s/localhost:%d/0.0.0.0:%d/" /clusterfuzz/src/go/testing/gcs/gcs.go
其中第二行是 docker 容器的 ip，是为了解决网站访问提交 job 压缩包时的上传地址问题
第三行是为了 9008 那个端口，变成 0.0.0.0
后续部分翻译自官方文档，还没亲测，大家可以先看看了解。
==========================================官方文档翻译分割线===============================================
启动执行机器人
官方命令：
python butler.py run_bot --name my-bot /path/to/my-bot
其中 my-bot 可以替换为自己喜欢的名称。我改成了 fuzzing-bot
$ python butler.py run_bot --name fuzzing-bot `pwd`/fuzzing-bot
执行成功后，可在前一步的管理员界面看到机器人状态。
可通过
tail -f `pwd`/fuzzing-bot/bot.log
查看机器人实时日志输出。
开始测试
官方给了一个例子，寻找 OpenSSL 的心脏滴血内存溢出漏洞。下面按照给出的步骤执行。
编译一个包含这个漏洞和已经带有 fuzz 插桩的 OpenSSL
# 下载并解压包含这个漏洞的 OpenSSL :
curl -O https://www.openssl.org/source/openssl-1.0.1f.tar.gz
tar xf openssl-1.0.1f.tar.gz
# 使用 AScan 和 fuzzer 插桩编译 OpenSSL:
cd openssl-1.0.1f/
./config
# 注意：$CC 必须指向 clang 二进制文件。简单地说，按照这个命令来写就对了
make CC="$CC -g -fsanitize=address,fuzzer-no-link"
cd ..
# 下载 fuzz target 和它的数据依赖:
curl -O https://raw.githubusercontent.com/google/clusterfuzz/master/docs/setting-up-fuzzing/heartbleed/handshake-fuzzer.cc
curl -O https://raw.githubusercontent.com/google/clusterfuzz/master/docs/setting-up-fuzzing/heartbleed/server.key
curl -O https://raw.githubusercontent.com/google/clusterfuzz/master/docs/setting-up-fuzzing/heartbleed/server.pem
# 编译可用于 ClusterFuzz 的 OpenSSL fuzz target （$CXX 需要指向一个 clang++ 二进制文件）:
$CXX -g handshake-fuzzer.cc -fsanitize=address,fuzzer openssl-1.0.1f/libssl.a \
openssl-1.0.1f/libcrypto.a -std=c++17 -Iopenssl-1.0.1f/include/ -lstdc++fs \
-ldl -lstdc++ -o handshake-fuzzer
zip openssl-fuzzer-build.zip handshake-fuzzer server.key server.pem
上传 fuzzer 到 ClusterFuzz
1、进入 Jobs 页面，点击【ADD NEW JOB】按钮
2、job 的各个输入框填写以下内容：
| 输入框名称 | 内容 |
|---|---|
| Name | libfuzzer_asan_linux_openssl |
| Platform | LINUX |
| Templates | libfuzzer engine_asan |
| Environment String | CORPUS_PRUNE = True |
3、把上一步打包的 openssl-fuzzer-build.zip 文件上传到 "Custom Build" 字段
4、点击【ADD】按钮，完成添加
5、点击【Select/modify jobs】，勾选 "libfuzzer_asan_linux_openssl" ，然后点击【SUBMIT】按钮
执行及查看结果
通过查看本地的机器人执行日志，可以发现 fuzz libFuzzer libfuzzer_asan_linux_openssl 这个字符串，代表目前 fuzz 测试已经在进行中了。
稍等一会，会在日志中发现一个堆栈信息和 AddressSanitizer: heap-buffer-overflow 出现在日志中。
再稍等一会，可以在 <> 页面看到一个标题为 "Heap-buffer-overflow READ{*}" 的测试用例，这个就是 ClusterFuzz 发现的心脏滴血漏洞了。
扩展性
从官方文档上看，上面的例子只是用到了引导式 fuzz ，ClusterFuzz 还支持可任意扩展的黑盒 fuzz ，可支持使用 Python 编写 fuzz 生成器。此次由于时间关系未能尝试，有兴趣的同学可以尝试一下。
同时官方的 local 文件夹中有看到 docker 运行相关的脚本，相信未来会支持通过 docker 运行，降低环境配置成本。
局限性
从官方文档中可以看到，被测试的软件需要在编译时插入一些桩用于检测异常，而这个方案目前仅支持 C/C++ ，且主要用于内存地址检测。而对于我们平时接触到的 Java/python/go 应用，没有提供对应的方案，需要另行扩展。
小结及展望
ClusterFuzz 正如其名，一个集群运行的 Fuzz 工具。它提供了执行机器人管理以及一个非常简便的管理界面，也做到了和研发流程无缝的接入，甚至更进一步地做到了 bug 自动创建及修复检测。
从小的地方看，它让模糊测试通过集群获得了更高的执行效率和问题发现效率。
从大的地方看，它提供的整体流程，包含了自动报 bug 和检测 bug 修复情况，让大家只在需要的时候感知到它的存在，正是目前大部分 CI 实践中欠缺的最后一公里路，缺陷的自动上报与修复检测，值得我们思考补全我们的 CI 流程。
虽然目前并未提供除 C/C++ 之外的完整解决方案，但相信按照其扩展性，扩展到支持更多的语言并不是难题。期望未来有更多的同学参与扩展这个工具，形成开箱即用的解决方案。
谢谢楼主分享
看完后还是不得要领，主要是不是很清楚，已有的模糊测试主要的作用是什么
今天倒是启动起来了，但是我找了好久没找到这个按钮。我看官方文档也是这么写的啊，哎。。
进入 Jobs 页面，点击【ADD NEW JOB】按钮
gongpx
回复
9002 端口的 web 可以访问，9000 端口的还是无法访问，提示 500 错误。
哎，不懂谷歌的东西这么难用，支持个代理这么难吗，系统设置了代理就出错了。而且启动服务指定了不安装依赖（都装好了）还是要去请求服务安装乱七八糟的东西，这还叫 “local instanse” 吗，哎
现在的情况是：
不设置系统代理，服务无法启动
设置了代理，服务可以启动，但是 9000 端口服务异常
fuzzing 的规则模型有没有介绍？
对于 go get 去拿 golang.org 域，就算已翻也未必会成功，一般 go 的资源在 gitlub 上都会有，直接 go get github 地址，就会直接拉下来编译好，然后改一下依赖路径，比较曲线救国，但也是我这里目前解决依赖问题的最直接方法
问题三可以设置代理 (注意要大写)
export HTTP_PROXY=http://
export HTTPS_PROXY=http://
搞这个快一天了没搞定，代理问题都搞死人了
================
最后安装成功了，但是无法启动本地 web 服务，一直报错
Running: python polymer_bundler.py (cwd='local')
| App Engine templates are up to date.
Clearing local datastore by removing local/storage.
Created symlink: source: /home/test/clusterfuzz/local/storage/local_gcs, target /home/test/clusterfuzz/src/appengine/local_gcs.
Traceback (most recent call last):
File "butler.py", line 282, in <module>
main()
File "butler.py", line 256, in main
command.execute(args)
File "src/local/butler/run_server.py", line 162, in execute
test_utils.setup_pubsub(constants.TEST_APP_ID)
File "/home/test/clusterfuzz/src/python/tests/test_libs/test_utils.py", line 308, in setup_pubsub
_create_pubsub_topic(client, project, queue['name'])
File "/home/test/clusterfuzz/src/python/tests/test_libs/test_utils.py", line 284, in _create_pubsub_topic
if client.get_topic(full_name):
File "/home/test/clusterfuzz/src/python/google_cloud_utils/pubsub.py", line 193, in get_topic
response = self._execute_with_retry(request)
File "/home/test/clusterfuzz/src/python/base/retry.py", line 88, in _wrapper
result = func(*args, **kwargs)
File "/home/test/clusterfuzz/src/python/google_cloud_utils/pubsub.py", line 108, in _execute_with_retry
return request.execute()
File "/home/test/clusterfuzz/src/third_party/googleapiclient/_helpers.py", line 130, in positional_wrapper
return wrapped(*args, **kwargs)
File "/home/test/clusterfuzz/src/third_party/googleapiclient/http.py", line 837, in execute
method=str(self.method), body=self.body, headers=self.headers)
File "/home/test/clusterfuzz/src/third_party/googleapiclient/http.py", line 162, in _retry_request
resp, content = http.request(uri, method, *args, **kwargs)
File "/home/test/clusterfuzz/src/third_party/google_auth_httplib2.py", line 198, in request
uri, method, body=body, headers=request_headers, **kwargs)
File "/home/test/clusterfuzz/src/third_party/httplib2/__init__.py", line 1694, in request
(response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
File "/home/test/clusterfuzz/src/third_party/httplib2/__init__.py", line 1434, in _request
(response, content) = self._conn_request(conn, request_uri, method, body, headers)
File "/home/test/clusterfuzz/src/third_party/httplib2/__init__.py", line 1390, in _conn_request
response = conn.getresponse()
File "/usr/lib/python2.7/httplib.py", line 1121, in getresponse
response.begin()
File "/usr/lib/python2.7/httplib.py", line 438, in begin
version, status, reason = self._read_status()
File "/usr/lib/python2.7/httplib.py", line 402, in _read_status
raise BadStatusLine(line)
httplib.BadStatusLine: ''
simple
回复
官方文档没提到 Fuzz 生成规则，只是大致提到了可以根据测试的代码覆盖率等数据自动优化生成的数据集。fuzz 生成器是是使用其它开源工具实现的，不是 ClusterFuzz 本身的内容。后续再细看下相关的资料。
陈恒捷
回复
我是 Ubuntu，开始配置的代理是小写的http_proxy，报错跟你一样的，也是 go 的依赖安装不上，后来搜了下，bazel 走代理，配置了之后就可以了
export HTTP_PROXY=http://ip:port
export HTTPS_PROXY=http://ip:port
另外发现配置了代理之后有时候还是连不上 google，我怀疑是 IPV6 的问题，所以我把系统的 IPV6 禁用了。。
现在我本地服务还是启动不了，提了一个 issue，现在还没人回。
细想了下，好像也不是邓爷爷说的。我还是改为 曾有人说 比较正规把。
terrychow
回复
babel 貌似不是到 go path 拿依赖的：
$ bazel run //local:create_gopath
INFO: Invocation ID: cbeff0a8-2bb4-4a56-ba63-8673567aa6f4
ERROR: /Users/hengjiechen/Develop/Python/clusterfuzz/src/go/cloud/db/BUILD.bazel:3:1: no such package '@org_golang_google_api//iterator': failed to fetch org_golang_google_api: 2019/02/20 22:19:22 get "google.golang.org/api": found meta tag vcs.metaImport{Prefix:"google.golang.org/api", VCS:"git", RepoRoot:"https://code.googlesource.com/google-api-go-client"} at http://google.golang.org/api?go-get=1
# cd .; git clone https://code.googlesource.com/google-api-go-client /private/var/tmp/_bazel_hengjiechen/169f8f67a11b1bde0973a54e7e2223f5/external/org_golang_google_api
Cloning into '/private/var/tmp/_bazel_hengjiechen/169f8f67a11b1bde0973a54e7e2223f5/external/org_golang_google_api'...
fatal: unable to access 'https://code.googlesource.com/google-api-go-client/': LibreSSL SSL_connect: SSL_ERROR_SYSCALL in connection to code.googlesource.com:443
2019/02/20 22:19:44 exit status 128
and referenced by '//go/cloud/db:go_default_library'
DEBUG: Rule 'org_golang_x_tools' modified arguments {"sha256": "2384fa91351a7414b643c5230422ce45f5aa2be8a82727609afd4e64e6973a30"}
ERROR: Analysis of target '//local:create_gopath' failed; build aborted: no such package '@org_golang_google_api//iterator': failed to fetch org_golang_google_api: 2019/02/20 22:19:22 get "google.golang.org/api": found meta tag vcs.metaImport{Prefix:"google.golang.org/api", VCS:"git", RepoRoot:"https://code.googlesource.com/google-api-go-client"} at http://google.golang.org/api?go-get=1
# cd .; git clone https://code.googlesource.com/google-api-go-client /private/var/tmp/_bazel_hengjiechen/169f8f67a11b1bde0973a54e7e2223f5/external/org_golang_google_api
Cloning into '/private/var/tmp/_bazel_hengjiechen/169f8f67a11b1bde0973a54e7e2223f5/external/org_golang_google_api'...
fatal: unable to access 'https://code.googlesource.com/google-api-go-client/': LibreSSL SSL_connect: SSL_ERROR_SYSCALL in connection to code.googlesource.com:443
2019/02/20 22:19:44 exit status 128
INFO: Elapsed time: 47.020s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded, 0 targets co\
FAILED: Build did NOT complete successfully (0 packages loaded, 0 targets co\
nfigured)
Fetching @org_golang_google_api; fetching 46s
Fetching @in_gopkg_yaml_v2; fetching 46s
Fetching ; Cloning 8dea3dc473e90c8179e519d91302d0597c0ca1d1 of https://g\
ithub.com/grpc/grpc-go 46s
Fetching ; Cloning aa810b61a9c79d51363740d207bb46cf8e620ed5 of https://g\
ithub.com/golang/protobuf 46s
我改了 src 文件夹下的 workspace 文件 可以绕过@org_golang_x_tools、@com_google_cloud_go、@org_golang_google_ap无法安装的情况
能不能截图个样例出来，到底怎么测试被测软件，否则这个环境搭建成本有点大
陈恒捷
回复
啊 不好意思没看见回复 我是在 ubuntu 下配的 我把 workspace 中的改成了如下：
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
name = "io_bazel_rules_go",
sha256 = "7be7dc01f1e0afdba6c8eb2b43d2fa01c743be1b9273ab1eaf6c233df078d705",
urls = ["https://github.com/bazelbuild/rules_go/releases/download/0.16.5/rules_go-0.16.5.tar.gz"],
)
http_archive(
name = "bazel_gazelle",
sha256 = "6e875ab4b6bf64a38c352887760f21203ab054676d9c1b274963907e0768740d",
urls = ["https://github.com/bazelbuild/bazel-gazelle/releases/download/0.15.0/bazel-gazelle-0.15.0.tar.gz"],
)
load("@io_bazel_rules_go//go:def.bzl", "go_rules_dependencies", "go_register_toolchains")
go_rules_dependencies()
go_register_toolchains()
load("@bazel_gazelle//:deps.bzl", "gazelle_dependencies", "go_repository")
gazelle_dependencies()
# Google Cloud client libraries.
go_repository(
name = "com_google_cloud_go",
commit = "7ceabc759e1fdbef87e9fda19ef24a5af9dde382",
importpath = "github.com/googleapis/google-cloud-go",
)
go_repository(
name = "org_golang_google_api",
commit = "a390a77b79400ac64702501e127022f5e2d986c9",
importpath = "github.com/googleapis/google-api-go-client",
)
go_repository(
name = "com_github_googleapis_gax_go",
commit = "b001040cd31805261cbd978842099e326dfa857b", # v2.0.2
importpath = "github.com/googleapis/gax-go",
)
go_repository(
name = "org_golang_x_oauth2",
commit = "9b3c75971fc92dd27c6436a37c05c831498658f1",
importpath = "github.com/golang/oauth2",
)
go_repository(
name = "org_golang_x_sync",
commit = "37e7f081c4d4c64e13b10787722085407fe5d15f",
importpath = "github.com/golang/sync",
)
go_repository(
name = "io_opencensus_go",
commit = "b7bf3cdb64150a8c8c53b769fdeb2ba581bd4d4b", # 0.18.0
importpath = "go.opencensus.io",
)
go_repository(
name = "io_opencensus_go_contrib_exporter_stackdriver",
commit = "2b7f4fc93daf5ec3048fa4fc1c15573466711a17", # 0.8.0
importpath = "contrib.go.opencensus.io/exporter/stackdriver",
)
go_repository(
name = "com_github_pkg_errors",
commit = "645ef00459ed84a119197bfb8d8205042c6df63d", # 0.8.0
importpath = "github.com/pkg/errors",
)
go_repository(
name = "com_github_shirou_gopsutil",
commit = "0f70a4a06f7a1039305ec9b3ba9c2800f8929fba", # v2.18.11
importpath = "github.com/shirou/gopsutil",
)
go_repository(
name = "in_gopkg_yaml_v2",
commit = "51d6538a90f86fe93ac480b35f37b2be17fef232", # v2.2.2
importpath = "gopkg.in/yaml.v2",
)
go_repository(
name = "org_golang_x_sys",
commit = "a9d3bda3a223baa6bba6ef412cb273f0fd163c05",
importpath = "github.com/golang/sys",
)
其实真正慢的原因并不在 pod 命令，而是在于 github 上的代码库访问速度慢，那么就知道真正的解决方案就是要加快 git 命令的速度。
我使用 Shadowsocks 代理，默认代理端口为 1086，配置好代理之后去终端输入 git 配置命令，命令如下
git config --global http.proxy socks5://127.0.0.1:1086
上面的命令是给 git 设置全局代理，但是我们并不希望国内 git 库也走代理，而是只需要 github 上的代码库走代理，命令如下
git config --global http.https://github.com.proxy socks5://127.0.0.1:1086
ps：如果要恢复/移除上面设置的 git 代理，使用如下命令
git config --global --unset http.proxy
git config --global --unset http.https://github.com.proxy
之前我弄 CocoaPods 的时候踩的坑
Zhperrrr
回复
改了以后，我这边是这样的：
| INFO: Elapsed time: 0.567s
| INFO: 0 processes.
| FAILED: Build did NOT complete successfully (0 packages loaded)
| ERROR: Build failed. Not running target
| FAILED: Build did NOT complete successfully (0 packages loaded)
| Return code is non-zero (1).
| Exit.
我遇到了一个新的问题，代码如下，请帮忙看看。
Traceback (most recent call last):
File "butler.py", line 282, in
main()
File "butler.py", line 256, in main
command.execute(args)
File "src/local/butler/run_server.py", line 162, in execute
test_utils.setup_pubsub(constants.TEST_APP_ID)
File "/clusterfuzz/src/python/tests/test_libs/test_utils.py", line 308, in setup_pubsub
create_pubsub_topic(client, project, queue['name'])
File "/clusterfuzz/src/python/tests/test_libs/test_utils.py", line 284, in _create_pubsub_topic
if client.get_topic(full_name):
File "/clusterfuzz/src/python/google_cloud_utils/pubsub.py", line 193, in get_topic
response = self._execute_with_retry(request)
File "/clusterfuzz/src/python/base/retry.py", line 88, in _wrapper
result = func(*args, **kwargs)
File "/clusterfuzz/src/python/google_cloud_utils/pubsub.py", line 108, in _execute_with_retry
return request.execute()
File "/clusterfuzz/src/third_party/googleapiclient/_helpers.py", line 130, in positional_wrapper
return wrapped(*args, **kwargs)
File "/clusterfuzz/src/third_party/googleapiclient/http.py", line 837, in execute
method=str(self.method), body=self.body, headers=self.headers)
File "/clusterfuzz/src/third_party/googleapiclient/http.py", line 162, in _retry_request
resp, content = http.request(uri, method, *args, **kwargs)
File "/clusterfuzz/src/third_party/google_auth_httplib2.py", line 198, in request
uri, method, body=body, headers=request_headers, **kwargs)
File "/clusterfuzz/src/third_party/httplib2/init.py", line 1694, in request
(response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
File "/clusterfuzz/src/third_party/httplib2/init.py", line 1434, in _request
(response, content) = self._conn_request(conn, request_uri, method, body, headers)
File "/clusterfuzz/src/third_party/httplib2/init_.py", line 1390, in _conn_request
response = conn.getresponse()
File "/usr/lib/python2.7/httplib.py", line 1136, in getresponse
response.begin()
File "/usr/lib/python2.7/httplib.py", line 453, in begin
version, status, reason = self._read_status()
File "/usr/lib/python2.7/httplib.py", line 417, in _read_status
raise BadStatusLine(line)
httplib.BadStatusLine: ''
在网上搜了一下，应该是 httplib.py 这个文件的问题，但是不知道该咋解决。
gongpx
回复
这样是不行的，我通过 ssh 连进去，不管配没配代理都是不成功的。
我只有通过 ubuntu 的系统界面 setting 配置代理，然后打开 terminal 才成功的。。
gongpx
回复
对于 29L：你往上边找找还有什么包没有装上
对于 27L：我之前也碰到了这个问题，我把终端代理关掉之后就没有这样的问题了
gongpx
回复
这个我提了个 issue，他们这样回复的
I has meet it,it is because it will access localhost port 9004 9003 9006 and so on when init,so the proxy server must on the same machine with clustfuzz server and igore localhost by proxy rules.
We have a note for this in documentation, see https://google.github.io/clusterfuzz/getting-started/local-instance/#running-a-local-server. list of ports needed are here - https://github.com/google/clusterfuzz/blob/master/src/local/butler/constants.py
"Note: the local instance may use ports other than 9000, such as 9008, for things like uploading files. Therefore, using the local instance may break if the needed ports are unavailable or if you can only access some of the needed ports from your browser (for example: because of port forwarding or firewall rules when accessing from another host)."
链接：https://github.com/google/clusterfuzz/issues/223
配置代理和不配置代理的错都不一样的，因为之前我是在虚拟机里面，ssh 进去的，今天给虚拟机安装了图形界面，直接在里面操作（在 setting 里面配置的代理），能启动起来打开管理界面了。
雨夜狂奔
回复
我现在是把代理都关了，然后就卡住了：
| App Engine templates are up to date.
Clearing local datastore by removing local/storage.
Created symlink: source: /clusterfuzz/local/storage/local_gcs, target /clusterfuzz/src/appengine/local_gcs.
卡在了这里，卡了 10 分钟了。。。
然后之前在这个模式下，挂代理，这个模式是能成功访问 google 的，但是就是会报上面那个错。。
坑三我通过设置 HTTP_PROXY 通过了
Created symlink: source: /clusterfuzz/local/storage/local_gcs, target /clusterfuzz/src/appengine/local_gcs.
这个问题通过图形化界面的网络代理设置，ignore 添加 localhost，然后取消终端的代理环境变量就通过了
9000 还是 500 错误
我是用代理运行的，运行：
sudo sh local/install_deps.sh
总是先下载 Python 的包，下好以后总是在一个地方卡住
+ bower@1.8.8
+ polymer-bundler@4.0.8
added 128 packages from 249 contributors in 125.917s
+ bower --allow-root install
bower app-layout#^2.0.1 not-cached https://github.com/PolymerElements/app-layout.git#^2.0.1
bower app-layout#^2.0.1 resolve https://github.com/PolymerElements/app-layout.git#^2.0.1
bower iron-collapse#^2.0.0 not-cached https://github.com/PolymerElements/iron-collapse.git#^2.0.0
bower iron-collapse#^2.0.0 resolve https://github.com/PolymerElements/iron-collapse.git#^2.0.0
bower iron-flex-layout#^2.0.0 not-cached https://github.com/PolymerElements/iron-flex-layout.git#^2.0.0
bower iron-flex-layout#^2.0.0 resolve https://github.com/PolymerElements/iron-flex-layout.git#^2.0.0
bower iron-form#^2.0.0 not-cached https://github.com/PolymerElements/iron-form.git#^2.0.0
bower iron-form#^2.0.0 resolve https://github.com/PolymerElements/iron-form.git#^2.0.0
bower iron-icon#^2.0.0 not-cached https://github.com/PolymerElements/iron-icon.git#^2.0.0
bower iron-icon#^2.0.0 resolve https://github.com/PolymerElements/iron-icon.git#^2.0.0
bower iron-icons#^2.0.0 not-cached https://github.com/PolymerElements/iron-icons.git#^2.0.0
bower iron-icons#^2.0.0 resolve https://github.com/PolymerElements/iron-icons.git#^2.0.0
bower iron-menu-behavior#^2.0.0 not-cached https://github.com/PolymerElements/iron-menu-behavior.git#^2.0.0
bower iron-menu-behavior#^2.0.0 resolve https://github.com/PolymerElements/iron-menu-behavior.git#^2.0.0
bower iron-meta#^2.0.0 not-cached https://github.com/PolymerElements/iron-meta.git#^2.0.0
bower iron-meta#^2.0.0 resolve https://github.com/PolymerElements/iron-meta.git#^2.0.0
bower iron-test-helpers#^2.0.0 not-cached https://github.com/PolymerElements/iron-test-helpers.git#^2.0.0
bower iron-test-helpers#^2.0.0 resolve https://github.com/PolymerElements/iron-test-helpers.git#^2.0.0
bower paper-button#^2.0.0 not-cached https://github.com/PolymerElements/paper-button.git#^2.0.0
bower paper-button#^2.0.0 resolve https://github.com/PolymerElements/paper-button.git#^2.0.0
bower paper-card#^2.0.0 not-cached https://github.com/PolymerElements/paper-card.git#^2.0.0
bower paper-card#^2.0.0 resolve https://github.com/PolymerElements/paper-card.git#^2.0.0
bower app-route#^2.0.0 not-cached https://github.com/PolymerElements/app-route.git#^2.0.0
bower app-route#^2.0.0 resolve https://github.com/PolymerElements/app-route.git#^2.0.0
bower iron-a11y-keys-behavior#^2.0.0 not-cached https://github.com/PolymerElements/iron-a11y-keys-behavior.git#^2.0.0
bower iron-a11y-keys-behavior#^2.0.0 resolve https://github.com/PolymerElements/iron-a11y-keys-behavior.git#^2.0.0
bower iron-ajax#^2.0.0 not-cached https://github.com/PolymerElements/iron-ajax.git#^2.0.0
bower iron-ajax#^2.0.0 resolve https://github.com/PolymerElements/iron-ajax.git#^2.0.0
bower paper-checkbox#^2.0.0 not-cached https://github.com/PolymerElements/paper-checkbox.git#^2.0.0
bower paper-checkbox#^2.0.0 resolve https://github.com/PolymerElements/paper-checkbox.git#^2.0.0
bower paper-dialog#^2.0.0 not-cached https://github.com/PolymerElements/paper-dialog.git#^2.0.0
bower paper-dialog#^2.0.0 resolve https://github.com/PolymerElements/paper-dialog.git#^2.0.0
bower paper-drawer-panel#^2.0.0 not-cached https://github.com/PolymerElements/paper-drawer-panel.git#^2.0.0
bower paper-drawer-panel#^2.0.0 resolve https://github.com/PolymerElements/paper-drawer-panel.git#^2.0.0
bower paper-dropdown-menu#^2.0.0 not-cached https://github.com/PolymerElements/paper-dropdown-menu.git#^2.0.0
bower paper-dropdown-menu#^2.0.0 resolve https://github.com/PolymerElements/paper-dropdown-menu.git#^2.0.0
bower paper-header-panel#^2.0.0 not-cached https://github.com/PolymerElements/paper-header-panel.git#^2.0.0
bower paper-header-panel#^2.0.0 resolve https://github.com/PolymerElements/paper-header-panel.git#^2.0.0
bower paper-icon-button#^2.0.0 not-cached https://github.com/PolymerElements/paper-icon-button.git#^2.0.0
bower paper-icon-button#^2.0.0 resolve https://github.com/PolymerElements/paper-icon-button.git#^2.0.0
bower paper-item#^2.0.0 not-cached https://github.com/PolymerElements/paper-item.git#^2.0.0
bower paper-item#^2.0.0 resolve https://github.com/PolymerElements/paper-item.git#^2.0.0
bower paper-listbox#^2.0.0 not-cached https://github.com/PolymerElements/paper-listbox.git#^2.0.0
bower paper-listbox#^2.0.0 resolve https://github.com/PolymerElements/paper-listbox.git#^2.0.0
bower paper-material#^2.0.0 not-cached https://github.com/PolymerElements/paper-material.git#^2.0.0
bower paper-material#^2.0.0 resolve https://github.com/PolymerElements/paper-material.git#^2.0.0
bower paper-spinner#^2.0.0 not-cached https://github.com/PolymerElements/paper-spinner.git#^2.0.0
bower paper-spinner#^2.0.0 resolve https://github.com/PolymerElements/paper-spinner.git#^2.0.0
bower paper-styles#^2.0.0 not-cached https://github.com/PolymerElements/paper-styles.git#^2.0.0
bower paper-styles#^2.0.0 resolve https://github.com/PolymerElements/paper-styles.git#^2.0.0
bower paper-tabs#^2.0.0 not-cached https://github.com/PolymerElements/paper-tabs.git#^2.0.0
bower paper-tabs#^2.0.0 resolve https://github.com/PolymerElements/paper-tabs.git#^2.0.0
bower paper-toggle-button#^2.0.0 not-cached https://github.com/PolymerElements/paper-toggle-button.git#^2.0.0
bower paper-toggle-button#^2.0.0 resolve https://github.com/PolymerElements/paper-toggle-button.git#^2.0.0
bower paper-toolbar#^2.0.0 not-cached https://github.com/PolymerElements/paper-toolbar.git#^2.0.0
bower paper-toolbar#^2.0.0 resolve https://github.com/PolymerElements/paper-toolbar.git#^2.0.0
bower paper-tooltip#^2.0.0 not-cached https://github.com/PolymerElements/paper-tooltip.git#^2.0.0
bower paper-tooltip#^2.0.0 resolve https://github.com/PolymerElements/paper-tooltip.git#^2.0.0
bower polymer#^2.1.0 not-cached https://github.com/Polymer/polymer.git#^2.1.0
bower polymer#^2.1.0 resolve https://github.com/Polymer/polymer.git#^2.1.0
bower test-fixture#^2.0.0 not-cached https://github.com/PolymerElements/test-fixture.git#^2.0.0
bower test-fixture#^2.0.0 resolve https://github.com/PolymerElements/test-fixture.git#^2.0.0
bower web-component-tester#^6.0.0 not-cached https://github.com/Polymer/web-component-tester.git#^6.0.0
bower web-component-tester#^6.0.0 resolve https://github.com/Polymer/web-component-tester.git#^6.0.0
bower iron-icon#^2.0.0 download https://github.com/PolymerElements/iron-icon/archive/v2.1.0.tar.gz
bower iron-collapse#^2.0.0 download https://github.com/PolymerElements/iron-collapse/archive/v2.2.1.tar.gz
bower iron-form#^2.0.0 download https://github.com/PolymerElements/iron-form/archive/v2.4.0.tar.gz
bower app-layout#^2.0.1 download https://github.com/PolymerElements/app-layout/archive/v2.1.1.tar.gz
bower iron-flex-layout#^2.0.0 download https://github.com/PolymerElements/iron-flex-layout/archive/v2.0.3.tar.gz
bower iron-icons#^2.0.0 download https://github.com/PolymerElements/iron-icons/archive/v2.1.1.tar.gz
bower iron-test-helpers#^2.0.0 download https://github.com/PolymerElements/iron-test-helpers/archive/v2.0.1.tar.gz
bower iron-meta#^2.0.0 download https://github.com/PolymerElements/iron-meta/archive/v2.1.1.tar.gz
bower paper-button#^2.0.0 download https://github.com/PolymerElements/paper-button/archive/v2.1.3.tar.gz
bower iron-menu-behavior#^2.0.0 download https://github.com/PolymerElements/iron-menu-behavior/archive/v2.1.1.tar.gz
bower iron-a11y-keys-behavior#^2.0.0 download https://github.com/PolymerElements/iron-a11y-keys-behavior/archive/v2.1.1.tar.gz
bower paper-card#^2.0.0 download https://github.com/PolymerElements/paper-card/archive/v2.1.0.tar.gz
bower app-route#^2.0.0 download https://github.com/PolymerElements/app-route/archive/v2.1.2.tar.gz
bower iron-ajax#^2.0.0 download https://github.com/PolymerElements/iron-ajax/archive/v2.1.3.tar.gz
bower paper-checkbox#^2.0.0 download https://github.com/PolymerElements/paper-checkbox/archive/v2.0.4.tar.gz
bower paper-dialog#^2.0.0 download https://github.com/PolymerElements/paper-dialog/archive/v2.1.1.tar.gz
bower paper-drawer-panel#^2.0.0 download https://github.com/PolymerElements/paper-drawer-panel/archive/v2.1.2.tar.gz
bower paper-header-panel#^2.0.0 download https://github.com/PolymerElements/paper-header-panel/archive/v2.1.0.tar.gz
bower paper-dropdown-menu#^2.0.0 download https://github.com/PolymerElements/paper-dropdown-menu/archive/v2.1.0.tar.gz
bower paper-icon-button#^2.0.0 download https://github.com/PolymerElements/paper-icon-button/archive/v2.2.1.tar.gz
bower paper-item#^2.0.0 download https://github.com/PolymerElements/paper-item/archive/v2.1.1.tar.gz
bower paper-listbox#^2.0.0 download https://github.com/PolymerElements/paper-listbox/archive/v2.1.1.tar.gz
bower paper-material#^2.0.0 download https://github.com/PolymerElements/paper-material/archive/v2.1.0.tar.gz
bower paper-spinner#^2.0.0 download https://github.com/PolymerElements/paper-spinner/archive/v2.1.0.tar.gz
bower paper-styles#^2.0.0 download https://github.com/PolymerElements/paper-styles/archive/v2.1.0.tar.gz
bower paper-tabs#^2.0.0 download https://github.com/PolymerElements/paper-tabs/archive/v2.1.1.tar.gz
bower paper-toolbar#^2.0.0 download https://github.com/PolymerElements/paper-toolbar/archive/v2.1.0.tar.gz
bower paper-tooltip#^2.0.0 download https://github.com/PolymerElements/paper-tooltip/archive/v2.1.1.tar.gz
bower paper-toggle-button#^2.0.0 download https://github.com/PolymerElements/paper-toggle-button/archive/v2.1.1.tar.gz
bower test-fixture#^2.0.0 download https://github.com/PolymerElements/test-fixture/archive/v2.0.1.tar.gz
bower polymer#^2.1.0 download https://github.com/Polymer/polymer/archive/v2.7.0.tar.gz
bower web-component-tester#^6.0.0 download https://github.com/Polymer/web-component-tester/archive/v6.5.0.tar.gz
bower iron-icon#^2.0.0 retry Download of https://github.com/PolymerElements/iron-icon/archive/v2.1.0.tar.gz failed with ETIMEDOUT, retrying in 1.5s
bower iron-collapse#^2.0.0 retry Download of https://github.com/PolymerElements/iron-collapse/archive/v2.2.1.tar.gz failed with ETIMEDOUT, retrying in 1.0s
bower iron-form#^2.0.0 retry Download of https://github.com/PolymerElements/iron-form/archive/v2.4.0.tar.gz failed with ETIMEDOUT, retrying in 1.2s
bower app-layout#^2.0.1 retry Download of https://github.com/PolymerElements/app-layout/archive/v2.1.1.tar.gz failed with ETIMEDOUT, retrying in 1.4s
bower iron-flex-layout#^2.0.0 retry Download of https://github.com/PolymerElements/iron-flex-layout/archive/v2.0.3.tar.gz failed with ETIMEDOUT, retrying in 1.9s
bower iron-icons#^2.0.0 retry Download of https://github.com/PolymerElements/iron-icons/archive/v2.1.1.tar.gz failed with ETIMEDOUT, retrying in 1.9s
bower iron-test-helpers#^2.0.0 retry Download of https://github.com/PolymerElements/iron-test-helpers/archive/v2.0.1.tar.gz failed with ETIMEDOUT, retrying in 1.1s
bower iron-meta#^2.0.0 retry Download of https://github.com/PolymerElements/iron-meta/archive/v2.1.1.tar.gz failed with ETIMEDOUT, retrying in 1.6s
bower paper-button#^2.0.0 retry Download of https://github.com/PolymerElements/paper-button/archive/v2.1.3.tar.gz failed with ETIMEDOUT, retrying in 1.4s
bower iron-menu-behavior#^2.0.0 retry Download of https://github.com/PolymerElements/iron-menu-behavior/archive/v2.1.1.tar.gz failed with ETIMEDOUT, retrying in 1.4s
bower iron-a11y-keys-behavior#^2.0.0 retry Download of https://github.com/PolymerElements/iron-a11y-keys-behavior/archive/v2.1.1.tar.gz failed with ETIMEDOUT, retrying in 1.6s
bower paper-card#^2.0.0 retry Download of https://github.com/PolymerElements/paper-card/archive/v2.1.0.tar.gz failed with ETIMEDOUT, retrying in 1.1s
bower app-route#^2.0.0
retry 不成功就终止了，有一样的吗？
yangxunpeng
回复
你用代理了吗？我用了，前面的 PY 的包还有一些其他的都没问题就卡在
bower iron-icon#^2.0.0 retry Download of https://github.com/PolymerElements/iron-icon/archive/v2.1.0.tar.gz failed with ETIMEDOUT, retrying in 1.5s
这里，暂时想换个代理试试，之前的不能用了，你有什么好思路吗？
雨夜狂奔
回复
用了代理，就是在 Ubuntu 的 Network->Setting 设置了代理。之前的下载是没有问题的，只卡在这个特殊的位置。
楼主，您好。
看到工具支持的一键安装环境不包含 centos 或者 redhat，是不是 工具暂不支持这些系统？
另外有个疑惑，win 下的支持是怎么做的，chrome 的测试应该也包含 win 的版本。
雨夜狂奔
回复
这个页面确实不好找，比如这里http://172.17.0.2:9000/testcases
在左上角那个三条横线的地方，点一下就会弹出一个面板，job,bot 等都可以在这里找到
waston
回复
坑 5 我已经解决了，其实不用改代码里各处的变量定义，修改脚本的启动参数即可，方式如下：
sed -i "s/--admin_port={admin_port}/--admin_port={admin_port} --admin_host=0.0.0.0 --host=0.0.0.0 --enable_host_checking=false /" /clusterfuzz/src/local/butler/run_server.py
sed -i "s/--env_var LOCAL_GCS_SERVER_HOST={local_gcs_server_host}/--env_var LOCAL_GCS_SERVER_HOST=http:\/\/172.17.0.2:9008/" /clusterfuzz/src/local/butler/run_server.py
sed -i "s/localhost:%d/0.0.0.0:%d/" /clusterfuzz/src/go/testing/gcs/gcs.go
其中第二行是我 docker 容器的 ip，是为了解决网站访问提交 job 压缩包时的上传地址问题
第三行是为了 9008 那个端口，变成 0.0.0.0
rywu
回复
应该要更久些，看机器性能吧，找到漏洞后，会在 testcases 页面看到问题记录，如下图：
点进去，可以看到调用栈等详细信息
已经可以跑起来谷歌的示例了，多谢楼主趟路，上面我也遇到的问题，我再一一回复下
另外如果是在 docker 里运行 clusterfuzz，docker run 命令还要增加--privileged，否则 LeakSanitizer 会报错：LeakSanitizer does not work under ptrace
lion-roadbike
回复
docker: read tcp 192.168.5.135:47390->104.18.121.25:443: read: connection reset by peer.
See 'docker run --help'.
run docker 碰到上述问题，请问你遇到过吗？谢谢！
楼主，您好，坑 4 您是如何解决的。我的报错提示是
Traceback (most recent call last):
File "butler.py", line 282, in
main()
File "butler.py", line 256, in main
command.execute(args)
File "src/local/butler/run_server.py", line 162, in execute
test_utils.setup_pubsub(constants.TEST_APP_ID)
File "/clusterfuzz/src/python/tests/test_libs/test_utils.py", line 308, in setup_pubsub
create_pubsub_topic(client, project, queue['name'])
File "/clusterfuzz/src/python/tests/test_libs/test_utils.py", line 284, in _create_pubsub_topic
if client.get_topic(full_name):
File "/clusterfuzz/src/python/google_cloud_utils/pubsub.py", line 192, in get_topic
request = self._api_client().projects().topics().get(topic=name)
File "/clusterfuzz/src/python/base/retry.py", line 88, in _wrapper
result = func(*args, **kwargs)
File "/clusterfuzz/src/python/google_cloud_utils/pubsub.py", line 89, in _api_client
discovery.DISCOVERY_URI.format(api='pubsub', apiVersion='v1'))
File "/clusterfuzz/src/third_party/httplib2/init.py", line 1694, in request
(response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
File "/clusterfuzz/src/third_party/httplib2/init.py", line 1434, in _request
(response, content) = self._conn_request(conn, request_uri, method, body, headers)
File "/clusterfuzz/src/third_party/httplib2/init_.py", line 1360, in _conn_request
raise ServerNotFoundError("Unable to find the server at %s" % conn.host)
httplib2.ServerNotFoundError: Unable to find the server at www.googleapis.com
这个工具现在有在实际使用吗？效果如何？
kilikilikjk
回复
时间久远，已经不大记得了。。。一般就 2 个方向，一个是读懂相关源码，源码肯定有线索；另一个是想办法搞定网络，抓个包就知道地址了。当时应该是看源码的。
liuyingchen
回复
我们部门刚开始实际使用，因为产品是有 ui 的，但 dock 里启动 ui 报错，所以折腾了一下在单做的 cmd 模式中绕过 ui 相关的。跑了两天发现 11 个（或组）问题，其中五个（或组）安全相关的，预计下周找相关开发分析确认并处理。
坑四按照作者的思路：
把 json 文件下载下来，放到 docker 里面，文件地址：https://www.googleapis.com/discovery/v1/apis/pubsub/v1/rest
修改 pubsub.py（路径：/clusterfuzz/src/python/google_cloud_utils/pubsub.py），修改内容看下图
就是把 http 请求函数注释掉，然后读取 json 文件（注意路径）
我这边测试可以在不连接 google 服务器的情况下启动 fuzzer（测试环境是作者提供的 docker）
不过作者是怎么找到这个文件地址的？
kilikilikjk
回复
原有界面项目（包含外部输入流程），按照示例实现一个 fuzz 命令行版（将输入流程接入 fuzz），并且上传 job 配置运行即可，如果命令行程序正确运行大概半天或一天后就会看到跑出来的案例。如果不正常就要看相关 log 来定位了：https://github.com/google/clusterfuzz/issues/670
lion-roadbike
回复
大佬们，这个报错怎么处理啊：ld: library not found for -lstdc++fs
TGDN-26684:clusterfuzz liz$ $CXX -g handshake-fuzzer.cc -fsanitize=address,fuzzer openssl-1.0.1f/libssl.a openssl-1.0.1f/libcrypto.a -std=c++17 -Iopenssl-1.0.1f/include/ -lstdc++fs -ldl -lstdc++ -o handshake-fuzzer -v
Apple clang version 11.0.0 (clang-1100.0.33.17)
Target: x86_64-apple-darwin19.4.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang" -cc1 -triple x86_64-apple-macosx10.15.0 -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name handshake-fuzzer.cc -mrelocation-model pic -pic-level 2 -mthread-model posix -mdisable-fp-elim -fno-strict-return -masm-verbose -munwind-tables -target-sdk-version=10.15 -target-cpu penryn -dwarf-column-info -debug-info-kind=standalone -dwarf-version=4 -debugger-tuning=lldb -ggnu-pubnames -target-linker-version 530 -v -resource-dir /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/11.0.0 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -I openssl-1.0.1f/include/ -I/usr/local/include -stdlib=libc++ -internal-isystem /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1 -Wno-framework-include-private-from-public -Wno-atimport-in-framework-header -Wno-extra-semi-stmt -Wno-quoted-include-in-framework-header -std=c++17 -fdeprecated-macro -fdebug-compilation-dir /Users/liz/Desktop/clusterfuzz -ferror-limit 19 -fmessage-length 80 -fsanitize-coverage-type=3 -fsanitize-coverage-indirect-calls -fsanitize-coverage-trace-cmp -fsanitize-coverage-inline-8bit-counters -fsanitize-coverage-pc-table -fsanitize=address,fuzzer,fuzzer-no-link -fsanitize-blacklist=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/11.0.0/share/asan_blacklist.txt -fsanitize-address-use-after-scope -fsanitize-address-globals-dead-stripping -fno-assume-sane-operator-new -stack-protector 1 -mdarwin-stkchk-strong-link -fblocks -fencode-extended-block-signature -fregister-global-dtors-with-atexit -fobjc-runtime=macosx-10.15.0 -fcxx-exceptions -fexceptions -fmax-type-align=16 -fdiagnostics-show-option -fcolor-diagnostics -o /var/folders/qr/f82hsc2958dcc6k8br085v200000gn/T/handshake-fuzzer-760ab7.o -x c++ handshake-fuzzer.cc
clang -cc1 version 11.0.0 (clang-1100.0.33.17) default target x86_64-apple-darwin19.4.0
ignoring nonexistent directory "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/v1"
ignoring nonexistent directory "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/local/include"
ignoring nonexistent directory "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/Library/Frameworks"
#include "..." search starts here:
#include <...> search starts here:
openssl-1.0.1f/include
/usr/local/include
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/11.0.0/include
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks (framework directory)
End of search list.
"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld" -demangle -lto_library /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/libLTO.dylib -no_deduplicate -dynamic -arch x86_64 -macosx_version_min 10.15.0 -syslibroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -o handshake-fuzzer /var/folders/qr/f82hsc2958dcc6k8br085v200000gn/T/handshake-fuzzer-760ab7.o openssl-1.0.1f/libssl.a openssl-1.0.1f/libcrypto.a -lstdc++fs -ldl -lc++ -L/usr/local/lib -lc++ /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/11.0.0/lib/darwin/libclang_rt.asan_osx_dynamic.dylib -rpath @executable_path -rpath /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/11.0.0/lib/darwin /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/11.0.0/lib/darwin/libclang_rt.fuzzer_osx.a -lc++ -lSystem /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/11.0.0/lib/darwin/libclang_rt.osx.a
ld: library not found for -lstdc++fs
clang: error: linker command failed with exit code 1 (use -v to see invocation)
多谢大佬们，想请教下：
按照上面说的，在本地进行 job 提交时没问题。但是远程提交就出现上传失败问题，不过远程进行其他操作没什么问题，防火墙已经关闭，不知道是否设置有误？
还有就是运行几个小时，一直无法找到漏洞，而在本地几秒就能找到漏洞，我是用 docker 运行的，也加了 privileged 选项。不知道有什么可能的问题？
mengnanpeter
回复
远程提交估计之前 sed 替换的语言要再改下，而且服务器和 bot 机器应该要分别替换。
比如 第二行：sed -i "s/--env_var LOCAL_GCS_SERVER_HOST=...
服务器那边不能用本地 ip 了，要用内网 ip；bot 机器那是改哪里不太确定 应该要指定服务器内网 ip 了，可以看看相关报错分析下，我这还是都在一台机器上。
另外运行几小时找不到可能是报错了（参见#75 楼），也可能是时间还要再久些。
lion-roadbike
回复
解决了，找到静态库的地址了，然后在命令里替换掉就 OK 了～
另外，大佬们，我在网页上配置 job 的时候，完全是 copy 的楼主例子里的数据，为什么会说 Invalid template name(s) specified？
就这个参数：
Templates libfuzzer engine_asan
不填这个参数的时候会报另外一个错，Failed to upload.上传失败
谷歌云服务，我们应该用不上。因为我们大量接口是不会再外网公开的。
此外，了解 FUZZ 的原理，与几个开发小伙伴开发一个针对内部接口 fuzz 的引擎才是王道。
另外 之前查过 好像也有 java 适配 fuzzer 的开源项目，不过可能也需要些开发成本
好用吗，在公司内部获取了什么样的成果
如果还是不好使，可以考虑用路由器翻
能测 所有 C 写的代码？
加精理由：快速实践新技术，前人踩坑，后人乘凉