Why choosing XCUITest framework over Appium for UI automation testing

I recently went for a Swift conference and UI automation testing was one of the subject. I already mentioned it with Appium in the past but I think it’s time to go back to it and explain why today I still prefer using Apple’s testing framework instead.

I discovered Appium about three years ago when I wanted to consolidate automation testing. At that time, it was a good alternative since Apple didn’t have an easy solution to maintain. However I already flagged at that time some limitation to Appium.

Over the years, Appium environment got better but nothing compare to XCUITest introduced since. When I feel Appium doesn’t still fully fit my purposes when native testing framework covers them all today.

Appium

First advantage about Appium is it’s an agnostic tool. It supports many code languages such as Python, PHP, Java, Javascript and more. It’s also very similar to Selenium since based on web driver, which make it a preferred tool for QA engineers.

It’s also pretty easy to setup and can be a good fit to keep one code base for test automation between an iOS and Android app.

However, it stays a very slow solution and if you link it to a continuous integration environment, it can take a while to actually make it efficient in the process. My second issue with it is the delay running the test, element detection stays very unstable and it can easily be missing if it didn’t wait enough time.

As close as you would like to have an iOS and Android apps, both platforms usually have different journey or UI components. At the end, you might have to support two testing code base anyway, one for each platform. If you already have to manage two projects (the mobile app itself and the ui test project), then it’s a lot to think about.

Finally, it seems to be a common idea that mobile engineers develop the features and QA engineers cover them with ui tests. I believe the responsibility shouldn’t be shared here. Same as unit tests, mobile developers should also add ui tests to their feature.

Who’s the best person to understand a feature than the developer who coded it at the first place?

XCUITest

On the other side, XCUITest is Apple native testing framework. It supports Objective-C as well as Swift, this is the advantage to put iOS developers to start using it.

I really like the fact it’s part of your same codebase, app and test sit together. It make it easy to developers to quickly test and see if unit tests and ui tests are still passing.

Since XCUITest is based on accessibility ids, it’s easy to implement and maintain on both side of the code. Creating an enum with your ids in the main bundle? Let’s include it to the test one to avoid hardcoded value.

Time to apply in my view

override func viewDidLoad() {
    super.viewDidLoad()

    self.departureButton.accessibilityIdentifier = AccessibilityIdentifiers.searchDeparture.rawValue
    self.arrivalButton.accessibilityIdentifier = AccessibilityIdentifiers.searchArrival.rawValue

    ...
}

Then finally we can easily create a test to detect the buttons

func testSearchButtons() {

    // launching the app
    let app = XCUIApplication()
    app.launch()

    // detect search buttons
    XCTAssertTrue(app.buttons[AccessibilityIdentifiers.searchDeparture.rawValue].waitForExistence(timeout: 5))
    XCTAssertTrue(app.buttons[AccessibilityIdentifiers.searchArrival.rawValue].waitForExistence(timeout: 5))
}

A good way to avoid flakiness in the code and make the test run faster is to use launch arguments. The best is that, once again, you can include the implementation on both side of your code: creating an enums with all launch arguments in the app, making it also accessible to the tests.

In my opinion, it also creates a team more independent: every iOS and QA engineers share the responsibility of testing, avoiding this specific separation in I’ve noticed sometimes. It also improve your continuous integration process: building faster with more robustness.

However, the main challenge might be to QA engineers to get included in that process: learning swift by implementing UI test can be a big challenge. It also can be a good way to learn new skills and eventually expand to iOS development.

In conclusion, XCUITest framework stays the best tool for UI automation testing to my opinion. It’s easy to maintain, fast to run, it avoids me flaky tests and also improves my continuous integration process. However it’s true that every project is different and if Appium already fits well your purpose, then there might be no reason to change from it.

Happy coding 🎉