在 testerhome 的两位大神推荐下，阅读并翻译了 12 Best Mobile App Testing Frameworks 这篇文章，了解了这 12 个框架。但里面介绍得实在少（基本都在 5 句话内），所以自己动手，通过框架官网介绍和示例代码比较来了解一下各个框架。

文章内含有一些自己观点，不一定完全正确，建议真正想了解的童鞋点击框架名称的链接去官网了解清楚

20150422 update: 增加总结部分

比较维度

1. 框架的目的及特色（框架自己的自我介绍）

2. 框架使用领域

3. 示例代码

4. 环境配置要求

1. Appium

框架的目的及特色：

Appium is an open source test automation framework for use with native, hybrid and mobile web apps. It drives iOS and Android apps using the WebDriver protocol.

使用领域：

Android，iOS 的原生应用、混合应用、web 应用的 UI 自动化

示例代码：

由于 Appium 支持多种语言，此处仅以 python 为例：

import os
from time import sleep

import unittest

from appium import webdriver

# Returns abs path relative to this file and not cwd
PATH = lambda p: os.path.abspath(
    os.path.join(os.path.dirname(__file__), p)
)

class SimpleAndroidTests(unittest.TestCase):
    def setUp(self):
        desired_caps = {}
        desired_caps['platformName'] = 'Android'
        desired_caps['platformVersion'] = '4.2'
        desired_caps['deviceName'] = 'Android Emulator'
        desired_caps['app'] = PATH(
            '../../../sample-code/apps/ApiDemos/bin/ApiDemos-debug.apk'
        )

        self.driver = webdriver.Remote('http://localhost:4723/wd/hub', desired_caps)

    def tearDown(self):
        # end the session
        self.driver.quit()

    def test_find_elements(self):
        el = self.driver.find_element_by_accessibility_id('Graphics')
        el.click()
        el = self.driver.find_element_by_accessibility_id('Arcs')
        self.assertIsNotNone(el)

        self.driver.back()

        el = self.driver.find_element_by_accessibility_id("App")
        self.assertIsNotNone(el)

        els = self.driver.find_elements_by_android_uiautomator("new UiSelector().clickable(true)")
        self.assertGreaterEqual(12, len(els))

        self.driver.find_element_by_android_uiautomator('text("API Demos")')


    def test_simple_actions(self):
        el = self.driver.find_element_by_accessibility_id('Graphics')
        el.click()

        el = self.driver.find_element_by_accessibility_id('Arcs')
        el.click()

        self.driver.find_element_by_android_uiautomator('new UiSelector().text("Graphics/Arcs")')


if __name__ == '__main__':
    suite = unittest.TestLoader().loadTestsFromTestCase(SimpleAndroidTests)
    unittest.TextTestRunner(verbosity=2).run(suite)

环境配置要求：

iOS Requirements

Mac OS X 10.7 or higher, 10.9.2 recommended
XCode >= 4.6.3, 5.1.1 recommended
Apple Developer Tools (iPhone simulator SDK, command line tools)
Ensure you read our documentation on setting yourself up for iOS testing!

Android Requirements

Android SDK API >= 17 (Additional features require 18/19)
Appium supports Android on OS X, Linux and Windows. Make sure you follow the directions for setting up your environment properly for testing on different OSes:

• linux
• osx
• windows

FirefoxOS Requirements

Firefox OS Simulator

2. Mobile Testing Framework

框架目的及特色：

The Mobile Testing Framework (MTF) is an alpha-state open source project to automate GUI/System tests for iPhone/iPad applications. MTF is based on the automation tool Sikuli which allows to interact with the User interface with python scripts and screenshots.（简而言之，把 sikuli 这种基于图像识别的测试框架引入到移动端）

使用领域：

iOS 的 GUI 及系统自动化测试

示例代码

目前仅支持 python，且需要 Configuration Files 配合编写：

python:

class Testsuite_001_Login(MobileTestCase):

# setUpClass is called before the test suite (optional)
#    @classmethod
#    def setUpClass(cls):
#       print "Setup Testsuite"

# tearDownClass is called after the test suite (optional)
#    @classmethod
#    def tearDownClass(cls):
#        print "Cleanup Testsuite"

# setUp is called before each test case (required!)
  def setUp(self):
        print "Test setup starts"
        startTest(self) #required by the framework
        startSimulator(self) #launches the simulator with the target app

# tearDown is called after each test case (required!)
def tearDown(self):
        endTest(self)
        print "Test cleanup done"

cfg file:

环境配置：

We assume you have Mac OS X Lion (either in a virtual machine or natively) installed. However, it should be similar for other versions of Mac OS X.

We assume you have XCode 4.3.3 (which comes with iOS 5.0 SDK) with the following installed (Check in XCode / Preferences / Downloads):
iOS Simulator 4.3 and 5.0 (and any other version in which you want to test your app) installed
Command Line Tools installed
We assume that you have installed macports ( http://www.macports.org/install.php )

除此之外，还需要另外装：Sikuli, ios-sim, GNU getopt 并配置 Dtrace settings

3. Calabash（写这个博文时 http://calaba.sh 官网有 404 错误，所以这里的链接是到 github repo）

框架目的及特色：

Calabash is an Automated UI Acceptance Testing framework that allows you to write and execute tests that validate the functionality of iOS and Android Apps.

框架使用领域：

手机原生/混合应用验收测试

示例代码

使用 Gherkin 和 cucumber：

Feature: Credit card validation.
  Credit card numbers must be exactly 16 digits.

  Scenario: Credit card number is too short
    Given I use the native keyboard to enter "123456" into text field number 1
    And I touch the "Validate" button
    Then I see the text "Credit card number is too short."

环境配置：

• Ruby 2.0 to 2.1.x.
• gems
• Bundler
• Xcode Command Line Tools(iOS)

4. KIF

框架目的及特色：

KIF, which stands for Keep It Functional, is an iOS integration test framework. It allows for easy automation of iOS apps by leveraging the accessibility attributes that the OS makes available for those with visual disabilities.

框架使用领域：

iOS 功能/集成测试

示例代码

object-c， KIF 支持 object-c 和 swift：

KIFUITestActor+EXAdditions.h

#import <KIF/KIF.h>

@interface KIFUITestActor (EXAdditions)

- (void)navigateToLoginPage;
- (void)returnToLoggedOutHomeScreen;

@end

LoginTestCase.m

#import "LoginTests.h"
#import "KIFUITestActor+EXAdditions.h"

@implementation LoginTests

- (void)beforeEach
{
    [tester navigateToLoginPage];
}

- (void)afterEach
{
    [tester returnToLoggedOutHomeScreen];
}

- (void)testSuccessfulLogin
{
    [tester enterText:@"user@example.com" intoViewWithAccessibilityLabel:@"Login User Name"];
    [tester enterText:@"thisismypassword" intoViewWithAccessibilityLabel:@"Login Password"];
    [tester tapViewWithAccessibilityLabel:@"Log In"];

    // Verify that the login succeeded
    [tester waitForTappableViewWithAccessibilityLabel:@"Welcome"];
}

@end

环境配置

Xcode，iOS 应用源码（项目文件，因为需要在项目文件内增加 target ）

5. Titanium Jasmine（根据 github commit 记录，这个框架已经 1 年没有更新过了）

框架目的及特色：

titanium-jasmine provides testing for Appcelerator’s Titanium Mobile applications using Pivotal’s Jasmine as the core testing framework.

框架使用领域：

Titanium Mobile 应用的自动化测试（本质上是增加一个测试模块到应用的 javascript 中执行测试）

示例代码

javascript

describe("titanium-jasmine", function() {
  it("should help you develop titanium applications", function() {
    expect(yourTests).toBe('awesome');
  });
});

环境配置（不确定，资料太少）：

Titanium Mobile SDK

6. Testing with Frankk

框架目的及特色：

Frank allows you to write structured text test/acceptance tests/requirements (using Cucumber) and have them execute against your iOS application.

框架使用领域：

iOS 应用验收/需求验证测试

示例代码

由于使用了 Cucumber，所以写法和 calabash 差不多：

    Feature: Login to the app
Scenario: Successful login
  Given I launch the app
  When I log in with a valid userid and password
  Then I am on the start view

环境配置:

iOS App：

Ruby, gem, Turn on Accessibility in OS X

Mac App：

Frank can be used to test 64-bit Mac apps on OS X 10.7 and 10.8. Due to the NDA covering the pre-release builds of 10.9, testing on 10.9 is not currently supported. If you need to run tests on 10.9 before the official release, see this thread.

The testing machine must have access for assistive devices turned on, as described in Getting Started.

7. Switchboard

框架目的及特色

Switchboard is a simple way to remote control your mobile application even after you shipped it to your users' devices. Use switchboard to

• Stage-rollout new features to users
• A/B-test user flows, messaging, colors, features
• anything you want to remote-control

Switchboard lets you control what happens in your app. Quick, easy, useful.

Switchboard segments your users consistently. Because user segmentation is based only on UUID that is computed once, the experience you switch on and off using Switchboard is consistent across sessions.

框架使用领域

Android 及 iOS 应用的 A/B 测试（即某个功能通过此框架可控制只提供给某些用户，通过对比使用此功能/不使用此功能的用户的状态，了解用户对此功能的满意度）

示例代码

分为 Client (嵌入到应用中) 和 server

Android Client:

//java

    Context myContext = this.getApplicationContext();
    String experimentName = "showSmiley";

    //get settings from Switchboard
    boolean isSmiling = Switchboard.isInExperiment(myContext, experimentName);

    //Switching code for testing
    if (isSmiling) //variant A
        showSmileyWelcomeMessage();
    else //variant B
        showFrownyFace();

iOS Client：https://github.com/KeepSafe/Switchboard/tree/master/client/ios/SwitchboardSample/SwitchboardSample

Server:

// php

    $manager = new SwitchboardManager($_GET);

    //put 50% of your users into the showSmiley A/B test
    $resultArray['showSmiley'] = $manager->turnOnBucket(0, 50);

    //return result array as JSON
    $manager->renderResultJson($resultArray);

环境配置

Android & iOS Client：加入 switchboard 依赖包，增加控制功能是否显示的代码

Server：安装 PHP

8. Flip Test

框架目的及特色

Easily create and manage A/B tests for iOS apps

框架使用领域

iOS A/B 测试

示例代码

原谅我在 官方网站 和官方博客 都没找到任何安装、使用文档，仅仅找到一个介绍视频（看起来是一个引导型的 web 页面，一步一步走就能把 fliptest 加到应用中），因此无法附上示例代码。

另外，官方网站有这一句话：

We're not quite ready to show FlipTest to the world yet, but please sign up if you would like to get notified when we get there!

按照上面的解读，这个框架其实没有公开。我已经订阅了邮件，看后面有没有收到它的使用文档。

各位读者如果有详细的文档麻烦通知我一下，我再把这部分补充上去。

环境配置

按照它的介绍，有 iOS 开发环境就足够了。

9. ReactiveUI

框架目的及特色

A MVVM framework that integrates with the Reactive Extensions for .NET to create elegant, testable User Interfaces that run on any mobile or desktop platform. Supports Xamarin.iOS, Xamarin.Android, Xamarin.Mac, WPF, Windows Forms, Windows Phone 8 and Windows Store apps.

框架使用领域

.NET 的 MVVM 框架（这是开发框架，不是用来测试的，同类的框架我知道的有 AngularJS。出现在文章中的原因应该是它有良好的可测性 ）。支持移动平台的 Xamarin.iOS, Xamarin.Android, Windows Phone 8 及桌面平台 Xamarin.Mac, WPF, Windows Forms, Windows Store apps

示例代码

应该是 C# ，不是很确定。。。

public class ColorChooserThatDoesntLikeGreen : ReactiveObject
{
  //
  // Declaring a read/write property
  //

  byte _Red;
  public byte Red {
    get { return _Red; }
    set { this.RaiseAndSetIfChanged(value); }
  }

  byte _Green;
  public byte Green {
    get { return _Green; }
    set { this.RaiseAndSetIfChanged(value); }
  }

  byte _Blue;
  public byte Blue {
    get { return _Blue; }
    set { this.RaiseAndSetIfChanged(value); }
  }

  //
  // Declaring a Property that's based on an Observable
  // 

  ObservableAsPropertyHelper<Color> _Color;
  public Color Color {
    get { return _Color.Value; }
  }

  ReactiveCommand OkButton { get; protected set; }

  public ColorChooserThatDoesntLikeGreen()
  {
    var finalColor = this.WhenAny(x => x.Red, x => x.Green, x => x.Blue, 
        (r,g,b) => Color.FromRGB(r.Value, g.Value, b.Value));

    finalColor.ToProperty(this, x => x.Color, out _Color);

    // When the finalColor has full green, the Ok button is disabled
    OkButton = new ReactiveCommand(finalColor.Select(x => x.Green != 255));
  }
}

环境配置

.Net 开发环境

10. Catjs

框架目的及特色

CATJS is an automation framework for web & mobile-web applications testing supports iOS, Android and any HTML5 Browser.

框架使用领域

web / 手机端 web 应用自动化测试 (实质上是一个 JS 测试框架)

示例代码

在官网的 Examples 里面。其中从@[scrap 到 ]@ 之间的就是测试代码，其他是应用代码

<div>
    <!--
            @[scrap
            @@name simpleSetText
                @@embed true
                @@jqm setText("textInput", "hello world");
            @@assert ok($('#textInput').val()=='hello world','textInput text == hello world')
        ]@
    -->
    <label for="textInput">Text input:</label>
    <input type="text" name="textInput" id="textInput" value="">
</div>

环境配置

直接运行在网页 JS 中，不需要额外的配置

11. YSO Mobile Security Framework

框架目的及特色

YSO Mobile Security Framework is an intelligent, all-in-one open source mobile application (Android/iOS) automated pen-testing framework capable of performing static and dynamic analysis.

We've been depending on multiple tools to carry out reversing, decoding, debugging, code review, and pen-test and this process requires a lot of effort and time.

YSO Mobile Security Framework can be used for effective and fast security analysis of Android APK/Android app source code/iOS app source code.

框架使用领域

多合一的 Android/iOS 手机应用渗透测试框架

示例代码

没找到。应该是不用写代码的，因为这类测试基本是通用型测试。测试完成后有个测试报告基本足够了。

环境配置

• Python 2.7
• JDK

12. Imagrium

框架目的及特色

Imagrium is a Jython framework for cross-platform testing of mobile applications basing on the image recognition method (Android and iOS application test automation software).

框架使用领域

基于图像识别进行 Android/iOS UI 自动化测试

示例代码

分为两部分： Pages 和 Tests

Pages

authPage = AuthPage.load(AppLauncher.box, self.settings)
fbAuthPage = authPage.signUpFb()
fbAuthPage.fillEmail(self.settings.get("Facebook", "email"))
fbAuthPage.fillPassword(self.settings.get("Facebook", "password"))
fbConfirmPage = fbAuthPage.login()
LobbyPage = fbConfirmPage.confirm() 

Tests

class FbAuthPage(Page):

    email = ResourceLoader([Resource.fbEmailFieldiOS, Resource.fbEmailFieldiOS_ru])
    password = ResourceLoader([Resource.fbPasswordFieldiOS, Resource.fbPasswordFieldiOS_ru])
    actionLogin = ResourceLoader([Resource.fbLoginBtniOS, Resource.fbLoginBtniOS_ru])

    def __init__(self, box, settings):
        super(FbAuthPage, self).__init__(box, settings)

        self.email = self.box
        self.password = self.box
        self.actionLogin = self.box

        self.settings = settings
        self.waitPageLoad()

        self.checkIfLoaded(['email', 'password'])

    def fillEmail(self, text):
        self.email.click()
        self.waitPageLoad()
        self.inputText(text)

环境配置

(1) Clone the git repo to the directory of your liking (versions for Win x64 and MacOS are available in the win and ios branches respectively).

(2) Install JDK 1.7.0_55+. The JAVA_HOME environment varialbe should be correctly set.

(3) (win) If you wish to use Imagrium for Windows, make sure the following software is installed and configured:

• Install VirtualBox 4.2+.

• Include the bin directory of VirtualBox in your PATH environment variable (otherwise, the vboxmanage utility will not be found).

• Install Genymotion 2.2+.

• Include the genymotion in your PATH environemnt variable (we need this to run the player utility).

• Install Android SDK.

• Include the platform-tools directory in your PATH environemnt variable (we need this to use the adb utility).

• Install Tesseract OCR.

• Install Apache Ant.

---

(3) (mac) If you wish to use Imagrium for MacOS, make sure the following software is installed and configured:

• ios-sim. Use npm, brew, or other ways to install it (visit the project and read the related docs). We use this utility to launch apps in the simulator.

• We use the Apple Script snippets to reset and rotate the Simulator, so copy the reset and rotateScreen scripts from the scripts directory in the repo root to the place where you can execute them.

• Ensure the user who launches the test run has a permission to execute these scripts (if this is your current user, run chmod +x reset rotateScreen ).

• IMPORTANT! Run reset and rotateScreen directly from Terminal beforehand to make sure that the system allows GUI manipulations from console. Run these scripts under the user who will later run tests. Accept the pop-ups asking the permission to manipulate the system through the Terminal app.

总结

框架名称	框架使用领域	测试脚本编写语言	环境配置	支持平台
Appium	Android，iOS 的原生应用、混合应用、web 应用的 UI 自动化	Java, Objective-C, JavaScript with Node.js (in promise, callback or generator flavors), PHP, Python, Ruby, C#, Clojure, Perl	iOS: Mac OS X 10.7 or higher, XCode >= 4.6.3, Apple Developer Tools; Android: Android SDK API >= 17; FirefoxOS: Firefox OS Simulator	Android, iOS
Mobile Testing Framework	iOS 的 GUI 及系统自动化测试	python	MAC OS X, XCode 4.3.3(iOS Simulator 4.3 and 5.0, Command Line Tools), macports, Sikuli, ios-sim, GNU getopt	iOS
Calabash	手机原生/混合应用验收测试	Gherkin + Cucumber	Ruby 2.0 to 2.1.x, gems, Bundler, Xcode Command Line Tools(iOS)	Android, iOS
KIF	iOS 功能/集成测试	object-c 或 swift	Xcode	iOS
Titanium Jasmine	Titanium Mobile 应用的自动化测试	javascript	Titanium Mobile SDK(不确定，资料太少)	Android, iOS
Testing with Frankk	iOS 应用验收/需求验证测试	Gherkin + Cucumber	iOS: Ruby, gem, Turn on Accessibility in OS X; Mac: have access for assistive devices turned on	iOS, MAC OS X
Switchboard	Android 及 iOS 应用的 A/B 测试	Android: Java; iOS: object-c	Android & iOS Client：加入 switchboard 依赖包，增加控制功能是否显示的代码; Server：安装 PHP	Android, iOS
Flip Test	iOS A/B 测试	object-c (只需增加 1 行代码)	XCode	iOS
ReactiveUI	.NET 的 MVVM 框架（开发用）	C#(不是很确定)	.Net 开发环境	Android, iOS, windows phone, Windows, MAC OS X
Catjs	web / 手机端 web 应用自动化测试	javascript（但是用的语法比较特殊）	浏览器	手机/桌面浏览器
YSO Mobile Security Framework	Android/iOS 手机应用渗透测试	无	Python 2.7, JDK	Android, iOS
Imagrium	基于图像识别进行 Android/iOS UI 自动化测试	python	JDK; windows 下：VirtualBox 4.2+, Genymotion 2.2+, Tesseract OCR, Apache Ant; Mac 下：ios-sim	Android, iOS