Macaca 记一次 XCTestWD 试用的情况,与 WDA 的对比

weamylady · 发布于 2017年09月30日 · 最后由 utopia 回复于 2017年10月12日 · 592 次阅读
本帖已被设为精华帖!

最近做了iOS远程真机的项目,使用ios-minicap + WDA来实现回显和操作。听说xctestWD功能要比WDA更稳定更快捷,于是进行了搭建使用,来评估是否应该把底层操作模块用XCTestWD来实现。

试用步骤:
1,从github下载源码进行编译,与WDA类似,其中遇到了一些坑,后面还是想办法解决了。
2,试用Xcode build 命令启动XCTestWD,给我感觉就是特别慢。后续掐表进行计算,WDA启动时间为11秒,XCTestWD启动时间为26秒……我的天!测了几次数据都区别不大,都是这个启动时间。

$ xcodebuild -project XCTestWD.xcodeproj \
           -scheme XCTestWDUITests \
           -destination 'platform=iOS,name=(your device name)' \
           XCTESTWD_PORT=8001 \
           clean test

3,试用了XCTestWD的截图速度,论坛上很多人都说截图速度快,我试了一下,的确比WDA快一些。时间小于0.5秒,wda大概需要1秒。
4,找了很多地方都没有找到XCTestWD的接口的说明,只能通过查看源码来判断它的web接口与WDA的区别。主要查看XCTestWD/XCTestWD/XCTestWDUITests/server/controllers目录下的各种.swift文件,比如在XCTestWDElementController.swift中可以看到以下代码:

static func routes() -> [(RequestRoute, RoutingCall)] {
    return [(RequestRoute("/wd/hub/session/:sessionId/element", "post"), findElement),
            (RequestRoute("/wd/hub/session/:sessionId/elements", "post"), findElements),
            (RequestRoute("/wd/hub/session/:sessionId/element/:elementId/element", "post"), findElement),
            (RequestRoute("/wd/hub/session/:sessionId/element/:elementId/elements", "post"), findElements),
            (RequestRoute("/wd/hub/session/:sessionId/element/:elementId/value", "post"), setValue),
            (RequestRoute("/wd/hub/session/:sessionId/element/:elementId/click", "post"), click),
            (RequestRoute("/wd/hub/session/:sessionId/element/:elementId/text", "get"), getText),
            (RequestRoute("/wd/hub/session/:sessionId/element/:elementId/clear", "post"), clearText),
            (RequestRoute("/wd/hub/session/:sessionId/element/:elementId/displayed", "get"), isDisplayed),
            (RequestRoute("/wd/hub/session/:sessionId/element/:elementId/attribute/:name", "get"), getAttribute),
            (RequestRoute("/wd/hub/session/:sessionId/element/:elementId/property/:name", "get"), getAttribute),
            (RequestRoute("/wd/hub/session/:sessionId/element/:elementId/css/:propertyName", "get"), getComputedCss),
            (RequestRoute("/wd/hub/session/:sessionId/element/:elementId/rect", "get"), getRect),
            (RequestRoute("/wd/hub/session/:sessionId/tap/:elementId", "post"), tap),
            (RequestRoute("/wd/hub/session/:sessionId/doubleTap", "post"), doubleTapAtCoordinate),
            (RequestRoute("/wd/hub/session/:sessionId/keys", "post"), handleKeys),
            (RequestRoute("/wd/hub/session/:sessionId/title", "get"), title),
            (RequestRoute("/wd/hub/session/:sessionId/homeScreen", "post"), homeScreen),
            (RequestRoute("/wd/hub/session/:sessionId/element/:elementId/doubleTap", "post"), doubleTap),
            (RequestRoute("/wd/hub/session/:sessionId/element/:elementId/touchAndHold", "post"), touchAndHoldOnElement),
            (RequestRoute("/wd/hub/session/:sessionId/element/:elementId/twoFingerTap", "post"), handleTwoElementTap),
            (RequestRoute("/wd/hub/session/:sessionId/touchAndHold", "post"), touchAndHold),
            (RequestRoute("/wd/hub/session/:sessionId/dragfromtoforduration", "post"), dragForDuration),
            (RequestRoute("/wd/hub/session/:sessionId/element/:elementId/pinch", "post"), pinch)]
}

看到大部分的url与WDA的接口url对比,就是多了个前缀/wd/hub。另外有些地方WDA的url中还需要/wda/,而此处没有了。
5,使用curl命令建立一个session,打开了ios中的设置 app:

curl -X POST '-H "Content-Type:application/json"' -d "{\"desiredCapabilities\":{\"bundleId\":\"com.apple.Preferences\",\"arguments\":[],\"environment\":{},\"shouldWaitForQuiescence\":true,\"shouldUseTestManagerForVisibilityDetection\":false,\"maxTypingFrequency\":60,\"shouldUseSingletonTestManager\":true}}" http://localhost:8200/wd/hub/session

手机上的响应很快,小与0.5秒就执行了打开设置App的操作,但是curl命令返回花了较多的时间:

可以看到iproxy中报了一个错误,暂时不太清楚是什么原因,可能是我使用iproxy的方式错了xctestwd应该用别的工具做代理?:

accepted connection, fd = 5
waiting for connection
Number of available devices == 1
Requesting connecion to device handle == 590 (serial: bb7787b7ba9d57bb6f9c84273d22fe3204d1e547), port 8001
run_ctos_loop: fd = 5
run_stoc_loop: fd = 5
recv failed: Resource temporarily unavailable
recv failed: Resource temporarily unavailable

6,使用点击步骤使用curl -X GET http://localhost:8200/wd/hub/sessions 命令获取 sessionId后,使用下面的命令点击:

curl -X POST '-H "Content-Type:application/json"' -d "{\"x\":\"325\",\"y\":\"350\"}" http://localhost:8200/wd/hub/session/「SessionId」/tap/0

使用点击步骤,手机上的响应速度与WDA基本一样。值得一提的是,如果我手动打开手机上的另一个App,tap步骤就会报错,XCUITestWD就会挂掉重启,日志如下:

    t = 10606.20s         Wait for app to idle
    t = 10627.05s     Find the Application "local.pid.988" 0x1742a4380
    t = 10627.05s         Snapshot accessibility hierarchy for local.pid.988
    t = 10627.28s     Tap Application "local.pid.988" 0x1742a4380[0.00, 0.00] -> (325.0, 350.0)
    t = 10627.28s         Wait for app to idle
    t = 10627.36s         Find the Application "local.pid.988" 0x1742a4380
    t = 10627.37s             Snapshot accessibility hierarchy for local.pid.988
    t = 10627.54s             Wait for app to idle
    t = 10627.63s         Synthesize event
    t = 10627.70s             Find the Application "local.pid.988" 0x1742a4380
    t = 10627.70s                 Snapshot accessibility hierarchy for local.pid.988
    t = 10627.92s         Wait for app to idle
    t = 10636.88s     Tap Application "local.pid.988" 0x1742a4380[0.00, 0.00] -> (325.0, 350.0)
    t = 10636.88s         Wait for app to idle
    t = 10643.17s             Unable to monitor animations
    t = 10649.35s             Unable to monitor event loop
    t = 10649.47s         Find the Application "local.pid.988" 0x1742a4380
    t = 10649.47s             Snapshot accessibility hierarchy for local.pid.988
2017-09-30 14:41:52.626646+0800 XCTRunner[1612:467713] *** Terminating app due to uncaught exception '_XCTestCaseInterruptionException', reason: 'Interrupting test'
*** First throw call stack:
(0x18d116fe0 0x18bb78538 0x18d116f28 0x10015b1f0 0x1001717c0 0x10019769c 0x100161d2c 0x100197484 0x100196400 0x10018579c 0x100161d2c 0x1001856dc 0x100181ab0 0x1001b35b8 0x100161d2c 0x1001b354c 0x10019bd78 0x106e0d63c 0x106df9424 0x106dff3d0 0x106e19400 0x106e18e44 0x18bfce9a0 0x18bfdf7a8 0x18bfce9a0 0x18bfd35e8 0x18d0c50c8 0x18d0c2ce4 0x18cff2da4 0x18db0ddb4 0x18db62704 0x106dfd584 0x106e17660 0x106e176a8 0x18d11ce80 0x18d0122c4 0x10015ca30 0x1001a0aac 0x10015c67c 0x100197dac 0x10015c430 0x10015cd80 0x100159f44 0x100159b64 0x100159d54 0x100159f44 0x100159b64 0x100159d54 0x100159f44 0x100159b64 0x100159d54 0x1001a5a14 0x1001688dc 0x1001a5894 0x100144f70 0x10019b6f8 0x18d0c530c 0x18d0c4b28 0x18d0c2998 0x18cff2da4 0x18ea5d074 0x1932adc9c 0x100018474 0x18c00159c)
libc++abi.dylib: terminating with uncaught exception of type _XCTestCaseInterruptionException
2017-09-30 14:42:39.901 xcodebuild[3177:2702920]  IDETestOperationsObserverDebug: Writing diagnostic log for test session to:
/Users/waterhuang/Library/Developer/Xcode/DerivedData/XCTestWD-fhjmeplezipylggqxqumqauziipm/Logs/Test/3588A363-4CDA-4851-BE06-5CA2CA6153B7/Session-XCTestWDUITests-2017-09-30_144239-nZIgnE.log
2017-09-30 14:42:39.902 xcodebuild[3177:2668008] [MT] IDETestOperationsObserverDebug: (0E195644-C42F-46D8-BDE7-F69DA1035F5F) Beginning test session XCTestWDUITests-0E195644-C42F-46D8-BDE7-F69DA1035F5F at 2017-09-30 14:42:39.902 with Xcode 8E3004b on target <DVTiOSDevice: 0x7fe74e0c2800> {
        deviceSerialNumber:         C8QP5BQHG5MP
        identifier:                 bb7787b7ba9d57bb6f9c84273d22fe3204d1e547
        deviceClass:                iPhone
        deviceName:                 iPhone
        deviceIdentifier:           bb7787b7ba9d57bb6f9c84273d22fe3204d1e547
        productVersion:             10.3.3
        buildVersion:               14G60
        deviceSoftwareVersion:      10.3.3 (14G60)
        deviceArchitecture:         arm64
        deviceTotalCapacity:        12241596416
        deviceAvailableCapacity:    7634079744
        deviceIsTransient:          NO
        ignored:                    NO
        deviceIsBusy:               NO
        deviceIsActivated:          YES
        deviceActivationState:      Activated
        isPasscodeLocked:           NO
        deviceType:                 <DVTDeviceType:0x7fe74b7de190 Xcode.DeviceType.iPhone>
        supportedDeviceFamilies:    (
    1
)
        applications:              {(
    <DTDKApplication: 0x7fe74e2c1e60>: CBS5 (/private/var/containers/Bundle/Application/01BEDD3A-2EEE-4C8F-ABA1-7D2CF18A410D/cbs.app),
    <DTDKApplication: 0x7fe74e3d4690>: 微博 (/private/var/containers/Bundle/Application/BFDEEA9D-1404-4E63-AE0B-101C27AFA4EE/Weibo.app),
    <DTDKApplication: 0x7fe74e2c3d20>: WebDriverAgentRunner (/private/var/containers/Bundle/Application/BF7B4137-B001-4710-9027-FF9A32A0FDA3/WebDriverAgentRunner-Runner.app),
    <DTDKApplication: 0x7fe74e23ccd0>: XCTestWDUITests (/private/var/containers/Bundle/Application/6D87C381-3D89-4732-A9AF-80C98161C3F0/XCTestWDUITests-Runner.app),
    <DTDKApplication: 0x7fe74e23dd50>: 智远一户通 (/private/var/containers/Bundle/Application/A4986F61-0691-4C39-A64C-1EC1DE174457/cms_yht_store.app),
    <DTDKApplication: 0x7fe74e29ea60>: 腾讯手机管家 (/private/var/containers/Bundle/Application/29B36462-E1DA-4E76-BB55-C2A9988EB118/MQQSecure.app),
    <DTDKApplication: 0x7fe74e33d1c0>: 招商银行 (/private/var/containers/Bundle/Application/62BD6AA5-929E-48B9-974C-F2EB709E3C4D/MPBBank.app),
    <DTDKApplication: 0x7fe74e23dd00>: QQ同步助手 (/private/var/containers/Bundle/Application/A2239219-79DC-4044-A205-42781540D062/QQPim.app),
    <DTDKApplication: 0x7fe74e28ac50>: Snapseed (/private/var/containers/Bundle/Application/7E1B5BDC-54ED-48E9-B6D6-0F0F8EDA6659/Snapseed.app),
    <DTDKApplication: 0x7fe74e3b2270>: 掌上生活 (/private/var/containers/Bundle/Application/BDC28B28-6423-47D4-B11F-4F808EBE737E/cmblife.app)
)}
        provisioningProfiles:      {(
    <DTDKProvisioningProfile 0x7fe74e122a40: UUID: 90b9d0ed-4efa-45eb-ab96-c90032c12c12, name: WDA, team: R3G94G7CQN (Water Huang), platform: iOS>,
    <DTDKProvisioningProfile 0x7fe74e146f60: UUID: 38d5f0ca-58eb-41e8-b713-bc42a61e35ea, name: iOS Team Provisioning Profile: *, team: R3G94G7CQN (Water Huang), platform: iOS>,
    <DTDKProvisioningProfile 0x7fe74df69210: UUID: 86c26768-bbbc-4698-982f-8386ad4a3b62, name: WDA, team: R3G94G7CQN (Water Huang), platform: iOS>,
    <DTDKProvisioningProfile 0x7fe74e219e90: UUID: b4c34aa9-f7a1-43bb-a067-eccddf362f7b, name: WDA, team: R3G94G7CQN (Water Huang), platform: iOS>,
    <DTDKProvisioningProfile 0x7fe74dfc69e0: UUID: 5b8bc8cb-6aea-4308-b75d-f417910a15de, name: cbsapp_distribution, team: U653R7WNB4 (China Merchants Bank Co.,Ltd.), platform: iOS>,
    <DTDKProvisioningProfile 0x7fe74df675f0: UUID: a66456ef-df52-43f2-a5c1-469ed536fe32, name: Appium wad, team: R3G94G7CQN (Water Huang), platform: iOS>,
    <DTDKProvisioningProfile 0x7fe74e38b5e0: UUID: 1256b74f-9a88-4842-9934-a5f7a3e587e8, name: iOS Team Provisioning Profile: *, team: R3G94G7CQN (Water Huang), platform: iOS>,
    <DTDKProvisioningProfile 0x7fe74df66540: UUID: 3a2aace7-4756-450f-ab03-368bc9565370, name: WDA, team: R3G94G7CQN (Water Huang), platform: iOS>,
    <DTDKProvisioningProfile 0x7fe74df650f0: UUID: 3dffc5dd-0093-4068-8185-b5d89c6ef244, name: iOS Team Provisioning Profile: *, team: R3G94G7CQN (Water Huang), platform: iOS>,
    <DTDKProvisioningProfile 0x7fe74e203070: UUID: 0a1dcba8-4389-43e9-8913-addb8e37ad09, name: iOS Team Provisioning Profile: *, team: R3G94G7CQN (Water Huang), platform: iOS>,
    <DTDKProvisioningProfile 0x7fe74e331500: UUID: 5b560df6-a952-4c5c-b604-24faf489caeb, name: WDA, team: R3G94G7CQN (Water Huang), platform: iOS>,
    <DTDKProvisioningProfile 0x7fe74df68190: UUID: f4f612fc-10c0-4740-8088-0985401e6edd, name: WDA, team: R3G94G7CQN (Water Huang), platform: iOS>
)}
        activityProgress:          -2
        activityTitle:             
        hasInternalSupport:        NO
        isSupportedOS:             YES
        developerDiskMountError:   (null)
(null)
    bootArgs:                  <unavailable>
        } (10.3.3 (14G60))
MDMCreateDeltaDirectory:1920 calling MDMDirectoryDiff with:
state->old_bundle: /var/folders/lq/q_5j5j1d2d5_v7w4bxmhgv5r0000gn/C/com.apple.DeveloperTools/All/Xcode/EmbeddedAppDeltas/15703a86a8b205ea28c81dc7e424278d/bb7787b7ba9d57bb6f9c84273d22fe3204d1e547/XCTestWDUITests-Runner.app
state->new_bundle: /Users/waterhuang/Library/Developer/Xcode/DerivedData/XCTestWD-fhjmeplezipylggqxqumqauziipm/Build/Products/Debug-iphoneos/XCTestWDUITests-Runner.app
state->dst_bundle: /var/folders/lq/q_5j5j1d2d5_v7w4bxmhgv5r0000gn/C/com.apple.DeveloperTools/All/Xcode/EmbeddedAppDeltas/XCTestWDUITests-Runner.app.qqHvOP/XCTestWDUITests-Runner.app_sparse.ipa/Payload//XCTestWDUITests-Runner.app, binaryDiff flag: FALSE
    dst_ipa: /var/folders/lq/q_5j5j1d2d5_v7w4bxmhgv5r0000gn/C/com.apple.DeveloperTools/All/Xcode/EmbeddedAppDeltas/XCTestWDUITests-Runner.app.qqHvOP/XCTestWDUITests-Runner.app_sparse.ipa
MDMDirectoryDiff_block_invoke:1473 calling writeDictToFile with: /var/folders/lq/q_5j5j1d2d5_v7w4bxmhgv5r0000gn/C/com.apple.DeveloperTools/All/Xcode/EmbeddedAppDeltas/XCTestWDUITests-Runner.app.qqHvOP/XCTestWDUITests-Runner.app_sparse.ipa/ManifestCache.plist
writeDictToFile:1278 ==== Successfully wrote Manifest cache to /var/folders/lq/q_5j5j1d2d5_v7w4bxmhgv5r0000gn/C/com.apple.DeveloperTools/All/Xcode/EmbeddedAppDeltas/XCTestWDUITests-Runner.app.qqHvOP/XCTestWDUITests-Runner.app_sparse.ipa/ManifestCache.plist
2017-09-30 14:42:01.570489+0800 XCTRunner[1631:481750] Running tests...
2017-09-30 14:42:02.548289+0800 XCTRunner[1631:481750] Continuing to run tests in the background with task ID 1

Restarting after unexpected exit or crash in XCTextWDRunner/testRunner(); summary will include totals from previous launches.

Test Suite 'Selected tests' started at 2017-09-30 14:42:02.968
Test Suite 'XCTestWDUITests.xctest' started at 2017-09-30 14:42:02.970
Test Suite 'XCTestWDUITests.xctest' failed at 2017-09-30 14:42:02.971.
     Executed 1 test, with 1 failure (0 unexpected) in 0.000 (0.001) seconds
Test Suite 'Selected tests' failed at 2017-09-30 14:42:02.973.
     Executed 1 test, with 1 failure (0 unexpected) in 0.000 (0.005) seconds
Failing tests:
    -[XCTextWDRunner testRunner()]
** TEST FAILED **

6,经过进一步测试,发现只要我这个sessionId是通过打开某个App来启动的,那么只能在这个App或者系统自带的页面(如拨号页面)可以进行点击,如果夸App进行操作,XCTestWD就会挂掉。由此可见建立的session是App相关的。但是!如果我随便填一个不存在的sessionId,那么在哪个应用中都可以进行点击,并不会报错T0T~~,例如下面的命令就可以在任何App中成功点击坐标:

curl -X POST '-H "Content-Type:application/json"' -d "{\"x\":\"325\",\"y\":\"350\"}" http://localhost:8200/wd/hub/session/1/tap/0

上面我的sessionID填了一个1。。。
7,在XCTestWD启动的时候启动iOS-minicap,发现同样也会挂掉,与WDA存在一样的问题。

结论:
1,XCTestWD的启动时间实在太慢,是WDA启动时间的两倍还多!就基于这一点我取消了替换WDA的想法~
2,点击、获取截图操作,时间花费与WDA相差不大,没有明显优势。
3,session的处理上与WDA不太一样,session不存在的时候也能进行操作,但是对于已存在的sesssion不能跨应用操作。

如果觉得我的文章对您有用,请随意打赏。您的支持将鼓励我继续创作!
共收到 3 条回复
104 seveniruby 将本帖设为了精华贴 09月30日 15:23
2457

请关注 https://github.com/macacajs/XCTestWD/issues/92 一起来打造最快最稳的框架

69ae3c

我之前也是使用的wda 确实在截图点击等操作上 性能几乎没有差别 但是在获取手机的xml结构的时候 差别太大了
wda基本上都是5s以上 xctestwd一秒以内 我做了一个实时操控平台 需要获取每个元素的具体信息 所以需要这个xml
结构 所以现在选的是xctestwd 不过xctestwd得操作都是有互斥锁的 所以你懂的

798276

这种技术贴越多越好

需要 登录 后方可回复, 如果你还没有账号请点击这里 注册