Why Appium? What distinguishes it from the competition?
The main advantage of Appium is that it’s cross-platform, meaning it will work on iOS, Android, Firefox, OS, etc. You can test in any language; and you don’t have to modify your app. Plus, you can use the selenium protocol that you’re already familiar with from web testing. And on the nice-to-have spectrum, Appium has a great community around it; plus it’s free and open source.
Here’s a quick look at the competition currently:
Instruments – Disadvantage? One language. All done in advance. Can’t talk to outside frameworks.
uiautomator – This works well; it also has instruments. But the language must be Java. tightly integration with app. Appium has more advanced features such as changing emulator language or xpath locators, however.
iOS driver – main disadvantage, small community. not widely used.
Selendroid – great project that appium supports. based on instruments, works on older android devices. more limited capabilities compared to uiautomator.
KIF – modifying your app, writing objective c into your app. rule #1 is don’t mess with the code. takes it one degree of separation from what the customers are going to see. there are some advantages to doing that. more tight integration, able to do some types of testing that appium can’t
Monkey Talk – low level. not the richness of appium
Does Appium work with mobile web and native apps?
Appium’s vision is to help automate everything mobile – including mobile web apps and native apps. It will continue to support Safari on iOS and Chrome on Android to use Appium to test mobile web and native apps. We are open to supporting other browsers.
Does it work with Sauce Labs? How to get started with Appium? This is the first seminar I am attending about this
Appium works on Sauce Labs. To get started checkout the documentation, tutorials, and sample code.
How should you handle flaky network connection simulation?
There definitely are tools. Proxies to change what your computer is doing.
Linux/OS X – ipfw – firewall rule tool
Windows – fiddler – javascript for writing rules
Appium also talks over TCP so don’t apply rules to 4444 or 4723.
Automating pre installed apps?
Android works well. UiAutomator allows you to test all apps on the device, on emulators and real devices. You can even go back and forth between apps.
On iOS, you can test in built in apps such as settings only on the simulator. On real devices, apple limits you to testing only your app.
Emulators vs real devices: which is better for testing?
Watch the GTAC presentation here
Check out how Google does their mobile testing here
Then ask yourself, what are you writing your automation for?
If it’s for business logic, simulators and emulators are probably fine.
If it’s for performance and crashes, consider real devices.
Overall, however, simulators and emulators are a highly effective way to save a lot of money.
*I’m joining a new company. How can I convince them to buy into Appium? *
You can use any language and test framework. It’s very easy to take existing selenium test infrastructure and use it for a proof of concept appium test cases. Fastest way to get started with mobile automation. Leverage existing languages and test frameworks that you use. Write a few small test cases then show it off as a demo. When people see appium running for the first time, it’s a pretty amazing experience.
What are the desired capabilities of Appium (versus Selenium)?
The capabilities were updated to match the mobile version of JSWP. Selenium wasn’t designed for mobile, so we worked with the Selenium project to define a new set of capabilities that support mobile.
platformName – ios/android
platformVersion – 7.1/4.4
deviceName – what kind of device we’re talking about iPhone Simulator/Nexus S
Documented well in the migration guide for appium 1.0.
If the same app is written for both platforms, how can I ensure a 100% cross-platform test?
The idea of writing one test for both platforms is the holy grail; but it requires the app be extremely similar. Even with xamarin and the same language for both, it depends on how similar the app is on both platforms.
It’s not about having 100%. Instead, have a small core that’s different, and then reuse the entire ecosystem about the test code. Everything can be the same except for a single definition file that encapsulates the differences.
Does Appium have significant advantages over webdriver for testing web apps on mobile devices?
Yes! Webdriver doesn’t provide drivers for mobile web apps or native apps. A tool like Appium is the only way you can automate mobile web apps currently.
Interestingly, the Selenium project recommends that you use a project like Appium in order to do this kind of automation. When you’re automating web apps using Appium, it’s just as though you’re using Selenium since there’s no difference in the code you write. Appium is simply providing the backend instead of the Selenium jar.
What is a short explanation as to what happens when an external application/activity is launched on Android/iOS?
It’s different on each platform. On Android, it’ll be as if the user launched it.
iOS has sandboxed enviornment, if your app opens a 3rd party app then the automation session will be shut down on you. That’s something that only Apple will be able to fix.
What is the release schedule for Appium?
The release process is becoming more structured. The github milestone tracker is being used with estimated dates. They will change.
The project planning is based around this list of issues assigned to a milestone. Feedback is used to prioritize.
How do new versions of Android/iOS impact Appium?
We’re trying to get better at using the beta. For example, we were using iOS 7.1 beta months before it was officially released. We’re able to anticipate issues before release and quickly add official support.
If you try Appium on a new release of Android/iOS, make sure to report issues.
What are Mavericks issues with Appium?
The latest OS X and Xcode release works well with Appium. If there are issues, make sure to open issues on GitHub.
How do I execute parallelization with Appium?
Use Sauce Labs to run in parallel on iOS. Sauce handles the virtualization of OS X for you. You can either maintain a bunch of OS X machines or let Sauce handle it for you.
With Android, it’s easier to run multiple emulators on your own system. If you’re looking to scale, then Sauce Labs is a big win.
Can you have multiple devices connected to one Appium server?
You can run one Appium server per device, on different ports. On iOS, it’s only one device per system due to Apple limitations.
Does Appium support Ruby, Python, C#, … etc?
Yes.
Appium is an open source source tool, but is there a support team that we can contact directly contact in case of issues?
Appium support is similar to how Selenium is supported. There’s a discussion group for questions and an issue tracker on GitHub for reporting bugs. If you have a commercial contract with a company that supports appium, such as Sauce Labs, then they have their own support channels.
Does Appium have the same approach as WebDriver/page objects?
Yes exactly. If you know how to write webdriver code and use page objects, then they’re applied in the same way. All of the concepts transfer.
Are there any hard button clicks supported in latest Appium versions for iOS devices ? I hope Android devices support this.
Appium makes the full UI Automation JavaScript API available as provided by Apple. You can click on buttons within the app. For special buttons such as home, there are work arounds. If you have a specific button in mind, I recommend asking on the appium discussion list.
Android has full support for clicking all buttons. The underlying methods are listed in the UiDevice documentation. To click the back button, for example, the standard selenium command is used. For arbitrary keys, there’s keyevent which is being renamed to press_keycode .
Are there plans to add OCR or image recognition support to appium?
There’s an open issue for FindByImage. Appium supports taking screenshots so you’re free to perform any image recognition based on that data.
Are there any tutorials available as to how to use an existing test automation system (in my case rspec/capybara) with appium?
Appium has code samples on GitHub that include an rspec example. There’s also a tutorial which covers Ruby and Java on appium.io. Documentation is available.
Are you considering hosting a conference about Appium in the near future? say 3 months down the road. Would be nice to have hands-on work shops etc.
If you’re interested in an Appium conference, read this post on the discussion list.
Are you planning to have scheduled releases, like every quarter or every six months, so we know when to expect Appium releases?
Upcoming appium releases along with estimated dates are tracked as milestones on GitHub. For the latest bug fixes, running from source is an option.
Can appium test cases run in parallel on different devices?
Running tests in parallel works fine for Android. On iOS, Apple has limited automation to work on one device at a time. The best way to run at scale with Android Emulators & iOS Simulators is to setup a selenium grid using virtual machines. Sauce Labs provides the easiest way to run tests in parallel on Android and iOS.
Is automation possible without having the Xcode of an app?
On iOS, you’re expected to have Xcode installed along with a simulator build of the app. A non-simulator build will not work on the iOS simulator. If Xcode isn’t installed then the automation will not work on iOS.
Can I use Appium to test web apps on mobile devices?
Yes. Appium fully supports testing web apps on mobile devices using Chrome on Android and Safari on iOS.
Can we use web application scripts used for Automation in Appium for native app as well?
Reusing test code is going to be a challenge when comparing web automation to native app automation. If you follow the page object pattern, then it’s entirely possible to share some code between the web and native app. Common technology such as test language, results reporting, and parallelization strategy can all be the same.
Can we do gestures on webview for Appium and not use webdriver since gestures are broken for iOS and Android?
The APIs for advanced gestures are currently limited to native apps. Standard WebDriver commands should work inside webviews for gestures.
Can you please provide a short explanation as to what happens when an external application/activity is launched during an Appium session (e.g., the contacts activity in android, or maps in iOS).
The idea behind UI automation is to test the app as a user would. The simple answer to this question is that it’s the same as if a user installed the app and then launched it. For more details, checkout the debug log of the Appium server as it contains exactly what’s going on.
Do I need to be an expert in Selenium WebDriver to effectively use Appium? In other words, is Selenium WebDriver experience a prerequisite for using Appium?
You need to learn WebDriver to use Appium effectively because that’s how Appium works. If you’re already familiar with uiautomator and UI Automation, then it’s possible to use Appium without understanding the details of WebDriver.
If you look at the sample code, the amount of WebDriver knowledge required is not at the expert level.
Do you have known issues when there are multiple devices connected on the same Appium server? We use 6 devices connect to the same Appium server.
To test with more than one Android device locally, you need to have one Appium server per device. For iOS, Apple limits automation to one device at a time. Sauce Labs enables parallel iOS automation by using OS X in a virtual machine.
Does Appium allow one to automate render.js webapps on mobile web for iOS and Android?
Yes, Appium supports automating web apps on iOS and Android.
Does Appium have significant advantages over WebDriver for testing straight up web apps on mobile devices?
Appium is one of the only ways available today to use WebDriver to test mobile apps on mobile devices. WebDriver alone doesn’t support mobile.
for Android, Appium doesn’t detect element id all the time, so I’ve had to use uiautomatorviewer. But uiautomatorviewer fails while using Appium. Any tips?
Make sure to end any existing Android automation sessions before using uiautomatorviewer. Once you have identified the elements using the viewer tool, then you can update the tests and run them via Appium.
Another workflow is to use the ruby_console to dynamically identify the elements and then interact with them before updating the test code.
Given that I have an app which makes online data exchanges through APIs, and as I want to write reproducible functional tests with Appium, how can I set up an environment to mock my app’s webservices calls in my testing scenarios?
Mocking an app’s webservice calls is a general testing issue and is unrelated to appium. I suggest researching appropriate mocking solutions for your language to see how others have accomplished this.
I have a good working knowledge of Selenium WebDriver 2.0 and of using Page Objects. Does Appium also has the same approach since Appium is using JSON driver behind the scenes?
Yes, the same approach is used.
Hi, I understand that Appium creates only one session for iOS as Apple UIAutomator does not support multiple iOS device support in parallel, whereas in Android, multiple devices are supported and hence each session maps with one devices. Is that correct?
Yes, running in parallel is currently limited to Android due to Apple’s limitations. A work around is to use Sauce Labs as they overcome the one device per machine via virtualization.
How is the support for scrolling going to improve? Currently it is very flaky to scroll up or down.
Properly writing scrolling code is a challenge. I’ve found using explicit waits helps resolve flakiness. On iOS it’s a bit easier because automation of invisible elements is possible. For Android, we’re limited to visible elements. I use complex_find on Android to scroll to elements. A better way is coming soon in Appium that’s easier to use.
Please send some examples of the multi action gesture API role in Appium?
Check out the unit tests for the various Appium client bindings to see examples of the gesture api.
Ruby bindings
Python bindings
Java bindings
JavaScript bindings
PHP bindings
C# bindings
How do I handle situations such as when Appium just fails, saying Javascript fail error (sendkeys throwing js error on line 68, failed, or my R&D says that Appium was not able to get the page source and that it may be because of a lot of data)?
Please report the issue on GitHub along with a way to reproduce the failure. On iOS I’m aware of a similar problem that’s being looked into. If the Appium devs can reproduce the problem then it’s much easier to fix.
How do I make static text (e.g. labels) appear in Appium inspector?
Properly making your app accessible, such as adding accessibility labels, is well documented by Google and Apple for Android and iOS respectively. I suggest consulting the documentation. Once your app is properly following the guidelines, then the labels will be visible to appium for automation.
How transparent would it be to associate Android and iOS devices/VMs/Emulators with our Selenium Grid infrastructure to use automated tests using Appium?
It’s entirely possible to set up local infrastructure. In my experience, this involves a lot of work so it’s better to use a service provider such as Sauce Labs.
I have seen user reporting lost of issue in Maverics OS using Appium …. how does the Appium project plan to address all these issues?
I have been using Appium on Mavericks without any issues. Make sure to use the newest version of OS X, Xcode, and Appium.
Is it possible to switch between native app and Safari when executing the workflow?
On iOS, this isn’t possible. The automation session is restricted to the app context and switching out will end the session. For Android, you could switch between the app and the Chrome browser without issue.
I use Firefox’s Responsive Web & UserAgent override to inspect Mobile Web Objects. Is there anything on the roadmap to inspect WebObjects directly from Appiums Inspector?
I recommend using the Chrome Dev tools instead of waiting for the Appium inspector to support web apps. The existing browser tools already work well for this purpose.
I’d love your recommendation for the best emulator for both iPhone and Android
This is highly specific to the business requirements of the app under test. I test on Android using a Nexus 7 and an iPhone Simulator. Sauce Labs supports a bunch of configurations to it’s up to the tester to decide what screen sizes and devices make sense.
I’m working on a mobile website project. Will Appium work for just mobile web, or is it specifically for hybrid and native apps?
Appium works well for mobile web, hybrid, and native apps. It’s not exclusive to any one of them.
If someone wanted to contribute to the most immediate need, what would that be? Features? Documentation? Quick Start Guide? Specific Language Usage Examples?
The most immediate need is documentation. The documentation files are hosted on GitHub and pull requests are welcome. Other areas to contribute include:
Additional examples for the sample code
Answer questions on the mailing list
Contribute to the tutorial Currently on Ruby and Java are covered.
If we want to be able to fully automate testing, we need to be able to respond to system prompts, like those that appear to allow the app to use location services or access contacts. Our experience with Appium thus far is that there’s no means to respond. Can you elaborate?
I suggest asking this question on the discussion list. I’m pretty sure that someone has figured out a work around.
In the future, will Appium support running multiple mobile test cases on simulators/emulators and real devices in parallel, instead of running tests one after another?
That’s already supported on Android. For iOS, Apple does not allow it. I suggest using a solution provider such as Sauce Labs that uses VMs to bypass this Apple limitation.
Inspector of Appium for Windows is much Flakey. Selendroid now doesnt gives Bounds/Size in PageSource.
For Selendroid, try their dedicated inspector tool
Is Appium 1.0.0 stable working for older Android like 4.2.x?
Appium works well on older Androids via Selendroid. uiautomator with Appium is available for API 17 and above.
Is Appium mainly for functional testing or integration testing? My company’s mobile device hits a webservice which returns a SQLite database, which we need to inspect for accuracy, once it’s been returned. Is this possible with Appium, even partially?
Appium is meant for end to end UI Testing, similar to how WebDriver is used to automate browsers. The situation described seems better suited for a web service unit test. I don’t see how a user of the app would inspect a SQLite database so appium doesn’t seem appropriate.
Is it always required that the Android app I’m testing have accessibility features (to locate the elements)? or I can use the simulator/recorder to locate those elements record tests for any app?
Accessibility features are not required. You can automate the app anyway however the other locator strategies will be brittle. The benefit of accessibility is the labels remain constant over the life cycle of the app. As the app changes, it’s better if tests continue working without having to update the locators.
Is there a plan to support testing on multiple iOS devices?
Apple hasn’t communicated about lifting this restriction. The current supported way is using Sauce Labs or another provider that handles parallelization.
Is there an easy way to setup Appium registered “devices” to Selenium GRID so that you can have a single point of access for tests?
A selenium grid is possible to setup. It’s not easy.
Just a suggestion, I also believe that we need more robust & rigorous UnitTest to be integrated with Appium CI, as the Appium release breaks a functionality or two while delivering another cool functionality.
I completely agree. I have experienced this issue and understand the frustration regressions cause. Proper unit testing with continuous integration is a work in progress. The goal is to have all Appium bindings in addition to the Appium server running robust unit tests via CI.
Last time I tried Appium, I had difficulty dealing with alerts coming from the OS or iTunes store (for payment automation). Does Appium 1.0 deal with OS/iTunes alerts better?
I’m not sure. If there’s a way to automate them with UI Automation JavaScript, then it’ll work with appium.
I need Appium/Node Logs to be consumed using a remote webdriver client. The WebDriver client is on remote m/c, so reading the log file doesn’t help much. Can we have a method to get log dumps via Appium-driver?
I believe there’s an open issue related to this. It makes sense to expose the logs via the Selenium log API. Currently supported logs include syslog (iOS), crashlog (iOS), and logcat (Android).
New to Appium, will soon use it. I briefly went over the multi action gestures. Are there any complex gestures that are not yet implemented? (or are still quirky/buggy)
So far the gestures seem to be working well.
One more question: Does Appium support C#, Java and Python languages?
Yes. Appium has client bindings for Ruby, Python, Java, JavaScript, PHP, and C#.
One of the big pain points for our QA is matching the implemented UI with the screen shots that our design team has put together. One guy in particular reached the laughing/crying stage imagining aloud the possibility of automation handling the comparison.
My personal opinion is that automatically matching design images to an app is not going to produce useful results. Appium supports screenshots so you’re welcome to try.
One of the requirements is XCode. Is Appium compatible with other IDEs, e.g. AppCode by Jetbrains?
Appium requires that Xcode is installed, not that you use it. Appium works with every IDE and can even be used without one.
Parallel execution always wonderful when it comes to faster testing. But iOS again some limitations. How are we going to address this?
The immediate solution is to use Sauce Labs. The longer term solution is to persuade Apple to stop limiting their automation technology.
Possibility to mock/stub webservices calls?
Mocks and Stubs are certainly possible. This is a general testing issue and is unrelated to appium.
Robotium can assert Android toasts by using something like waitForText(). Can Appium do the same thing?
uiautomator does not support toasts. Selendroid does.
A Selenium grid basically helps to redirect test cases to right devices … will this help with iOS as we are not able to run the iOS devices in parallel due to Apple limitation.
You could setup your own grid or use an existing solution provider such as Sauce Labs. Grid is not a magic solution to Apple’s limitation on parallelization.
Should we also depreciate/discourage usage of Android version less than 4.2?(Coz UIAutomator came at 4.2, and Selendroid is not stable to use)
I haven’t had good luck with Selendroid either so I only use uiautomator.
I’m attending this webinar to learn about Appium as someone who has never used it to automate testing. I enjoy being able to view screen shots in Sauce when I check my Selenium results… can Appium tests provide the same kind of screen shot results?
Yes, Sauce provides the same support for Appium including screenshots and videos. Locally, you would have to write the code to take the screenshots. I have open sourced a screen recorder for OS X that works with the iOS Simulator and Android emulator.
We had trouble writing tests that interact with tableViews, despite the fact that they are a common element in iOS gui development. We’ve seen that this is on the list of known issues. Is addressing this on the roadmap in the near future?
I suggest opening an issue on GitHub and include an example that reproduces the problem. My understanding is that tableViews should not have problems on iOS.
*Does Sauce Labs provide training for using Appium with Sauce? *
That’s a question to ask the sales team at Sauce Labs. I am working on screencast training that will be available to everyone for free.
Can you speak to whether/when you plan to address how to respond to prompts? Any workarounds available now?
I suggest posting on the discussion list. There are various ways to respond to prompts.
We had trouble migrating tests that ran on Firefox into tests that ran on Internet Explorer. Is it really that easy to migrate our tests to mobile with different browsers?
Appium enables you to automate mobile browsers. Chrome is a different browser from Safari so there will be differences. Browser automation has been successfully done for many years now. It’s entirely possible to test on multiple browsers.
We have the behavior that tests run locally stable and sometimes fail on the Appium server. For us this is not reproducible and could not determine why we have this instability. Do you know that? How do fix?
Flakiness can be caused by a large number of issues:
test written incorrectly
timing issues
app bugs
appium server bugs
appium client bugs
underlying automation bugs
emulator/simulator bugs
device specific bugs
network connectivity
environment configuration
software versions
If you post on the discussion list with specifics, then someone may be able to provide guidance.
We start our Appium tests with Jenkins and have in one example job 5 tests. From time to time we have 6 test result? Do you know why? It it a Appium issue?
This isn’t an Appium issue. There’s something broken with your test infrastructure if you’re randomly having an extra test result.
What are the changes app developers need to make in-app in order for visible property to be true in Appium inspector?
On Android, everything is always visible if it’s in the inspector. On iOS, if the element is displayed on screen then it should be visible -although sometimes it isn’t. Unfortunately there’s not much you can do to control the visibility attribute.
What are your future goals for automating native apps for Samsung TV and Apple TV using Appium?
I’m not aware of specific goals to support TV automation. If the TV runs Android then it may work.
What is the best test reporting plugin for mobile that would fit with Appium (JUnit) to be included in CI?
JUnit has numerous plugins for various CI solutions that enable reporting. This isn’t Appium specific or unique to mobile. I suggest researching online to find out what others are using.
What is the main difference between Selandroid and Appium?
Selendroid only supports Android and uses an instrumentation approach. Appium allows automation of iOS and Android. Appium supports Selendroid as one option and also allows using the more modern uiautomator technology.
What native application support does Appium offer?
Appium supports fully automating native apps on Android and iOS. On Android there’s a choice of Selendroid or uiautomator. On iOS, UI Automation JavaScript is used behind the scenes.
Please suggest an operating system on which one should primarily test when launching Android – I.E. Linux, OSX, Windows…?
OS X
When I run Appium, I got the following error, “Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure.” What can I do on my end to fix this?
This is a common issue. I suggest posting to the mailing list if it happens frequently.
When I tried Appium I found it was difficult to pick objects; it seems I need extensive xpath knowledge. Do you have plan to create an object library layer?
The Ruby binding has a helper library so you can use a generic find command that’ll work on almost anything. There are specific helpers for buttons, textfields, text, and alerts. I recommend not using XPath on Android as it’s flaky due to a known Android problem. With all the selector methods available, finding elements is very easy. First select which attribute you’re using to find the element then select the best selector.
When might Ruby users expect switch_to() to be implemented? We have many hybrid apps that need context switching.
The new Ruby bindings have implemented context switching via set_context. Known issues are in the process of being fixed; these will improve the reliability of context switching.
Will there be a Appium conference soon?
App-solutely! For information about the Appium conference, please read this mailing list thread.
Will there ever be integrated support for CI though Sauce Labs?
Sauce Labs integrates with CI providers such as CloudBees. All CI solutions should be able to integrate with Sauce Labs. There are even plugins available.
You mentioned earlier that you are working on ‘Getting Started’ materials and tutorials… any idea on when you might be ready to release these and where should I look to find them?
These are already available on Appium.io. Check out the documentation, tutorials, and sample code.
You mentioned you now now support Android hybrid apps without Selenium. Which version of Appium supports this, and are there docs that show how to do this?
On Android API 19, Appium is able to connect directly to webviews via chromedriver when using uiautomator. The client libraries have support for listing the webview context and switching into it and back to the native app.