Appium JAVA Appium 多点触控问题—求指导

南琛 · 2015年10月09日 · 最后由 南琛 回复于 2015年10月10日 · 2005 次阅读

小弟最近在研究图案解锁,想利用多点触控来解决问题,但是实际过程中发现一个奇怪的问题,待小弟慢慢说来。
JAVA CLIENT 版本 3.0.0

JAVA 调用代码如下

final TouchAction gesture = new TouchAction((MobileDriver) driver)
                .press(point0.getX(), point0.getY())
                .moveTo(point1.getX(), point1.getY())
                .moveTo(point2.getX(), point2.getY())
                .moveTo(point3.getX(), point3.getY()).release();

        gesture.perform();

实际上给出的 4 个点坐标值分别为:

(226, 756)
(226, 1071)
(226, 1386)
(539, 1386)

查看 appium debug log 发现:

> info: --> POST /wd/hub/session/92bcec80-2500-40a3-ae79-2223d4cd17dd/touch/perform {"actions":[{"action":"press","options":{"x":226,"y":756}},{"action":"moveTo","options":{"x":226,"y":1071}},{"action":"moveTo","options":{"x":226,"y":1386}},{"action":"moveTo","options":{"x":539,"y":1386}},{"action":"release","options":{}}]}
> info: [debug] Pushing command to appium work queue: ["element:touchDown",{"x":226,"y":756}]
> info: [debug] [BOOTSTRAP] [debug] Got data from client: {"cmd":"action","action":"element:touchDown","params":{"x":226,"y":756}}
> info: [debug] [BOOTSTRAP] [debug] Got command of type ACTION
> info: [debug] [BOOTSTRAP] [debug] Got command action: touchDown
> info: [debug] [BOOTSTRAP] [debug] Display bounds: [0,0][1080,1776]
> info: [debug] [BOOTSTRAP] [debug] Performing TouchDown using element? false x: 226, y: 756
> info: [debug] [BOOTSTRAP] [debug] Returning result: {"value":true,"status":0}
> info: [debug] Pushing command to appium work queue: ["element:touchMove",{"x":452,"y":1827}]
> info: [debug] [BOOTSTRAP] [debug] Got data from client: {"cmd":"action","action":"element:touchMove","params":{"x":452,"y":1827}}
> info: [debug] [BOOTSTRAP] [debug] Got command of type ACTION
> info: [debug] [BOOTSTRAP] [debug] Got command action: touchMove
> info: [debug] Responding to client with error: {"status":29,"value":{"message":"The coordinates provided to an interactions operation are invalid.","origValue":"Coordinate [x=452.0, y=1827.0] is outside of element rect: [0,0][1080,1776]"},"sessionId":"92bcec80-2500-40a3-ae79-2223d4cd17dd"}

发送的请求是正确的,第一个点也能正确点击到,第二点的 moveTo 的坐标居然变成了: ["element:touchMove",{"x":452,"y":1827}] 有木有啊。。。

仔细一看,发现是把 moveTo 的坐标值跟第一次 press 的坐标值相加之后得到的结果,导致超出了坐标系,报了个错

org.openqa.selenium.interactions.InvalidCoordinatesException: The coordinates provided to an interactions operation are invalid. (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 45 milliseconds

尝试第二种方法:MultiTouchAction

MultiTouchAction multiTouchAction = new MultiTouchAction((MobileDriver) driver);

        TouchAction action0 = new TouchAction((MobileDriver) driver).press(point0.getX(), point0.getY());
        TouchAction action1 = new TouchAction((MobileDriver) driver).moveTo(point0.getX(), point0.getY());
        TouchAction action2 = new TouchAction((MobileDriver) driver).moveTo(point0.getX(), point0.getY());
        TouchAction action3 = new TouchAction((MobileDriver) driver).moveTo(point0.getX(), point0.getY()).release();
        multiTouchAction.add(action0).add(action1).add(action2).add(action3).perform();

appium debug log:

> info: --> POST /wd/hub/session/59049042-efab-4b94-be86-546f14138a93/touch/multi/perform {"actions":[[{"action":"press","options":{"x":226,"y":756}}],[{"action":"moveTo","options":{"x":226,"y":756}}],[{"action":"moveTo","options":{"x":226,"y":756}}],[{"action":"moveTo","options":{"x":226,"y":756}},{"action":"release","options":{}}]]}
> info: [debug] Pushing command to appium work queue: ["performMultiPointerGesture",{"actions":[[{"action":"press","time":0.005,"touch":{"x":226,"y":756}}],[{"action":"moveTo","time":0.005,"touch":{"x":226,"y":756}}],[{"action":"moveTo","time":0.005,"touch":{"x":226,"y":756}}],[{"action":"moveTo","time":0.005,"touch":{"x":226,"y":756}}]]}]
> info: [debug] [BOOTSTRAP] [debug] Got data from client: {"cmd":"action","action":"performMultiPointerGesture","params":{"actions":[[{"action":"press","time":0.005,"touch":{"x":226,"y":756}}],[{"action":"moveTo","time":0.005,"touch":{"x":226,"y":756}}],[{"action":"moveTo","time":0.005,"touch":{"x":226,"y":756}}],[{"action":"moveTo","time":0.005,"touch":{"x":226,"y":756}}]]}}
> info: [debug] [BOOTSTRAP] [debug] Got command of type ACTION
> info: [debug] [BOOTSTRAP] [debug] Got command action: performMultiPointerGesture
> info: [debug] [BOOTSTRAP] [debug] Returning result: {"value":"OK","status":0}
> info: [debug] Responding to client with success: {"status":0,"value":"OK","sessionId":"59049042-efab-4b94-be86-546f14138a93"}
> info: <-- POST /wd/hub/session/59049042-efab-4b94-be86-546f14138a93/touch/multi/perform 200 137.629 ms - 76 {"status":0,"value":"OK","sessionId":"59049042-efab-4b94-be86-546f14138a93"}
> info: --> POST /wd/hub/session/59049042-efab-4b94-be86-546f14138a93/touch/perform {"actions":[{"action":"press","options":{"x":226,"y":756}},{"action":"moveTo","options":{"x":226,"y":1071}},{"action":"moveTo","options":{"x":226,"y":1386}},{"action":"moveTo","options":{"x":539,"y":1386}},{"action":"release","options":{}}]}
> info: [debug] Pushing command to appium work queue: ["element:touchDown",{"x":226,"y":756}]
> info: [debug] [BOOTSTRAP] [debug] Got data from client: {"cmd":"action","action":"element:touchDown","params":{"x":226,"y":756}}
> info: [debug] [BOOTSTRAP] [debug] Got command of type ACTION
> info: [debug] [BOOTSTRAP] [debug] Got command action: touchDown
> info: [debug] [BOOTSTRAP] [debug] Display bounds: [0,0][1080,1776]
> info: [debug] [BOOTSTRAP] [debug] Performing TouchDown using element? false x: 226, y: 756
> info: [debug] [BOOTSTRAP] [debug] Returning result: {"value":true,"status":0}
> info: [debug] Pushing command to appium work queue: ["element:touchMove",{"x":452,"y":1827}]
> info: [debug] [BOOTSTRAP] [debug] Got data from client: {"cmd":"action","action":"element:touchMove","params":{"x":452,"y":1827}}
> info: [debug] [BOOTSTRAP] [debug] Got command of type ACTION
> info: [debug] [BOOTSTRAP] [debug] Got command action: touchMove
> info: [debug] [BOOTSTRAP] [debug] Display bounds: [0,0][1080,1776]
> info: [debug] [BOOTSTRAP] [debug] Returning result: {"value":"Coordinate [x=452.0, y=1827.0] is outside of element rect: [0,0][1080,1776]","status":29}
> info: [debug] Responding to client with error: {"status":29,"value":{"message":"The coordinates provided to an interactions operation are invalid.","origValue":"Coordinate [x=452.0, y=1827.0] is outside of element rect: [0,0]

出现了同样的问题,发送的指令看似正确,但是执行的时候却错误了。。。T_T

求大神指教。。@monkey @chenhengjie123

共收到 4 条回复 时间 点赞

遇到类似的问题,我的报错是:
info: [debug] [BOOTSTRAP] [debug] Display bounds: [0,0][1080,1920]
info: [debug] [BOOTSTRAP] [debug] Display bounds: [0,0][1080,1920]
info: [debug] [BOOTSTRAP] [debug] Swiping from [x=270.0, y=960.0] to [x=900.0, y=960.0] with steps: 70
info: [debug] Responding to client with error: {"status":13,"value":{"message":"An unknown server-side error occurred while processing the command.","origValue":"The sw
ipe did not complete successfully"},"sessionId":"9141e0e4-141c-48b1-9f9b-777c1ec61785"}
info: <-- POST /wd/hub/session/9141e0e4-141c-48b1-9f9b-777c1ec61785/touch/perform 500 16.698 ms - 208
info: [debug] [BOOTSTRAP] [debug] Returning result: {"status":13,"value":"The swipe did not complete successfully"}

已经在 github 报 issue

https://github.com/appium/appium/issues/5703

@tobecrazy 应该是 appium 底层的 bug,目前我已经找到实际的操作结果是,touchation 连续操作的时候,第二个点的坐标是相对于第一个按下点的坐标,所以传值的时候传相对坐标就可以解决这个问题, 如果在官方不修复的情况下可以将就着这么用,目前问题已经通过错上加错的方式解决了。。

3楼 已删除

找到原因了,appium 的 moteTo 在 android 的实现是当传入的是纯坐标时,把坐标值认为是 offset。

相关代码在 https://github.com/appium/appium/blob/74c4ec1a20c1f20cd2ef0b821d314b02957c82bb/lib/devices/android/android-controller.js 中的 androidController.parseTouch 方法。

关键代码:

...
// expects absolute coordinates, so we need to save these as offsets
        // and then translate when everything is done
        options.offset = true;
        options.x = (gesture.options.x || 0);
        options.y = (gesture.options.y || 0);

        touchStateObject = {
          action: gesture.action,
          options: options,
          timeOffset: 0.005,
        };
        touchStateObjects.push(touchStateObject);
        done();
...
if (state.options.offset && prevPos) {
        // the current position is an offset
        state.options.x += prevPos.x;
        state.options.y += prevPos.y;
      }
...

解决方案是给 moveTo 传入 element 对象(你帖子正文的例子里就可以这么用),而不是传入坐标值。

PS:Java-client 的方法说明里有提到这个:
https://github.com/appium/java-client/commit/d7874ca6b93f3e72f612402beea61964033ce9fe#diff-970f723bf2aa8111d5817a3de004cc13

@chenhengjie123 谢谢。 但是实际情况是,我只能找到一个图案解锁的对象,通过算法计算出 9 个点的位置然后绘制解锁图案,所以每次 moveTo 只能传坐标值,现在通过传入相对与 press 点的坐标值得到了解决。

需要 登录 后方可回复, 如果你还没有账号请点击这里 注册