Understanding UI Testing using iOS, Xcode 9 and Swift

Xcode provides a fully-featured, scriptable UI Testing framework. A key to using the framework is understanding its architecture and how to best leverage its capabilities.

Understanding an Xcode UI Test

When you create a new project in Xcode, the new project wizard asks if you’d like to Include Unit Tests, and whether you’d like to Include UI Tests.

Xcode Test Target Selection

One might wonder — is a UI Test not a Unit test? If not, then what is it?

Actually, these checkboxes and their outcomes are primarily there to inform Xcode which targets to create within your project. Each checkbox, when checked, generates a different type of test target in your project.

The fundamental differences between an Xcode Unit Test and an Xcode UI Test:

  • Unit Tests are used to test that source code generates expected results. For example: ensuring that a function, when passed a specific parameter, generates some expected result.
  • UI Tests test that a user interface behaves in an expected way. For example: a UI Test might programmatically tap on a button which should segue to a new screen, and then programmatically inspect whether the expected screen did load, and contains the expected content.

Both Unit Tests and UI Tests support full automation, and enable regression testing of applications over their lifecycle.

Generally speaking, an Xcode Unit Test exercises and evaluates your Swift code, but does not inspect the impact is has on the User Interface, while an Xcode UI Test evaluates the UI behavior in response to actions, but does not inspect your code.

As always, these are generalized statements that have exceptions. It is certainly possible to get some insight into the UI from a (code) Unit Test, and to get some insight into the state of your code objects from a UI Test. But the tools are designed to be used according to this generalized statement.

Example of a UI Test

Before examining the architecture and implementation of UI Test, let’s take a look at a finished test in operation. The user story for this test is as follows:

On the first screen, the user can select a cell within a table view, which opens a second form showing the selected value in a label. The user can then key in a new value, in a text box beneath the original label. When the user subsequently returns to the first form, the new value will be shown in the Table View.

If a QA tester were to manually check this process they would do the following sequence:

  1. Launch the app
  2. Tap on a row
  3. Observe the table view row text is on the second form when it loads
  4. Type in a new value in the text field
  5. Press the back button
  6. Observe the value they typed has replaced the original text in the table view

The manual testing process would look as follows (this is an animated .gif — if using the Medium app, you may need to open this page in a browser to view the animation).

UI Test Process

Wouldn’t it be nice if we could automate this process so our QA tester didn’t have to repeat this process manually before every release? That’s exactly what UI Testing is for — and we’ll walk through how to automate this test process!

UI Testing Architecture

Before digging into the code, it’s important to understand how the Xcode UI Test framework operates. By understanding how the UI Tests access and manipulates your UI components, you’ll be able to make your UI easy to build tests for.

As with Unit Tests (the ones that exercise your source code), XCode uses XCTest to run your UI Tests. But how does the XCTest code know how to inspect and manipulate the UI that you designed in Storyboards and/or Swift code?

To gain access to your UI at runtime, XCTest uses metadata exposed by iOS Accessibility. This is the same technology used to enable iOS to read your screen to blind and low vision users, for example. At runtime, XCTest iterates over your UI controls, looking for Accessibility properties such as accessibilityIdentifier and accessibilityLabel to find the UI components you’d like XCTest to tap on, change or inspect as part of your UI Test.

While it’s possible to design UI Tests without doing any preparation of Accessibility metadata in your app — and you’ll find many examples on the Internet that do this — you can maintain better control and predictability in UI Tests by planning for UI Tests in advance, and preparing Accessibility metadata in the UI. Similarly, if you’re retrofitting UI Tests to an existing application, you should consider retrofitting Accessibility metadata as part of the process.

UI Test Recording

Xcode’s UI Test suite provides an easy way to get started implementing a UI Test: the Record UI Test button.

To begin recording a UI Test:

  1. Create a new UI Test function in the UI Test target source .swift file (assuming you created a UI Test target when you created your project — or added it later)
  2. Place the editing cursor within the empty test function
  3. Press the Record UI Test button below the source code editing pane

Xcode will compile and run the application using the debug device (i.e. simulator). Then, just walk through the test sequence on the simulator (or other debug device). When you’re finished, stop the debug session. Xcode will have created a set of commands to re-create the UI experience during the recording. In the case of the test sequence outlined above, the following code would be generated:

func testChangeTableRowText() {
   let app = XCUIApplication()
   app.tables["MyTable"].staticTexts["Fourth Row"].tap()
   let newvalueTextField = app.textFields["newValue"]
   newvalueTextField.tap()
   let app2 = app
   app2.buttons["shift"].tap()
   newvalueTextField.typeText("Some new value")
   app.navigationBars["UITestingDemo.DetailView"]
                                     .buttons["Back"].tap()
}

Great! Xcode has generated all the command needed to re-run the same UI Test process we did by hand. This is a boon to our test design productivity, and gives us a great start. But it’s not perfect, and not a production ready test yet. There are some deficiencies:

  1. There are some messy aspects, such as the line let app2 = app. We wouldn’t have written the code this way ourselves — the app object created at line 1 obviously can be used throughout the test function.
  2. The reference to staticTexts[“Fourth Row”] in line 2 of the function assumes that the contents of the UITableView cells will always be the same. What if it won’t? This is a case where preparing the Accessibility metadata can help make a more robust test. I’ll cover this shortly.
  3. The auto-generated code causes the test to operate, but nothing here is evaluating whether the outcomes of the test were successful or not. Xcode can’t create this part of the test — we have to do this ourselves.

Preparing the Accessibility Metadata

In Line 2 of the auto-generated code, Xcode inserted this line:

app.tables["MyTable"].staticTexts["Fourth Row"].tap()

In english, this command means:

Within the array ofUITableView objects within the current UIView, find a UITable with the key MyTable. Then, search all the UILabel controls within that UITable and find a UILabel having a text value “Fourth Row”. Then tap on that UILabel.

There are two key references XCTest uses to find UI elements here:

  1. The “Fourth Row” UILabel — the UILabel text value displayed on the 4th UITableViewCell in the UITable
  2. The UITableView with a key of “MyTable” — huh? Where did that key come from?

Let’s consider the second item. In this case, I had previously assigned the text “MyTable” as the accessibilityIdentifier for the UITable on the first UIView. This was done in the viewDidLoad() function of that UIView’s UIViewController, like so:

override func viewDidLoad() {
   super.viewDidLoad()
   tableView.accessibilityIdentifier = "MyTable"
}

Every UIView can have an accessibilityIdentifier, as well as other Accessibility properties. For the purposes of UI Testing, you’ll be most interested in accessibilityIdentifier and accessibiltyLabel.

Example of Accessibility Properties

When a UIView has either an accessibilityIdentifier or an accessibilityLabel, it can be queried within a UI Test by using that string as a key. For example, this table could be accessed within a UI Test in either of these ways:

let tableView = app.tables.containing(.table, identifier: "MyTable")
let tableView = app.tables["MyTable"]

By using Accessibility metadata in this way, you can create a more robust UI Test — one not dependent on the content of the text in controls. Instead, the controls can be accessed by dictionary key values you define and control. But you do need to make the effort to assign keys in order to use them!

Note: while UIView objects can be queried using either accessibilityIdentifier or accessibilityLabel, it’s usually better to use accessibiltyIdentifier. accessibilityLabel is the property iOS Accessibility uses to access the text to be read to a blind or low vision user, and could change at runtime for controls that have updatable text properties.

How to Set accessibilityIdentifiers

Setting the accessibilityIdentifier for a UIView-based object can be done in several ways. The most common are as follows:

Using the Interface Builder Identity Inspector

Some UI elements support setting of Accessibility properties within IB Identity Inspector. For example, the UILabel on the first form of our test solution has its accessibilityIdentifier set to “labelIdentifier” directly within the predefined IB field.

Setting the accessibilityLabelIdentifier for a UILabel

Using a User-Defined Runtime Attribute

For UI elements that wouldn’t normally be read to a blind or low vision end-user, Interface Builder won’t have predefined Accessibility property fields. But you can still add them at Interface Builder design time using the User Defined Runtime Attributes dictionary editor on the Identity Inspector.

In this case, I’ve moved the UITableView’s accessibilityIdentifier from the UIViewController’s viewDidLoad() method into the Interface Builder storyboard editor. The resulting UI Test works exactly the same way — but with less code to maintain.

Setting accessibilityLabelIdentifier using Runtime Attributes

Using Code

As mentioned earlier, every UIView-based class has accessibility properties, and those properties can be set at runtime.

override func viewDidLoad() {
   super.viewDidLoad()
   tableView.accessibilityIdentifier = "MyTable"
}

All three of these methods have the same effect . Which is best depends on best practices within your team. Some prefer to reduce code by configuring UI in Interface Builder, others prefer to do all UI design in code. UI Testing supports both scenarios equally well.

Inspecting UI Elements During the Test

Recall earlier that I recorded the steps for the test — but I didn’t actually test for anything! Let’s wrap this job up by adding the actual tests, and use accessibilityIdentifier properties where possible.

Searching for UIView elements

Recall that Xcode wrote the following statement to find the UITableView using its accessibilityIdentifier:

let tableView = app.tables["MyTable"]

This is the most concise shorthand method, but I want to point out there’s more than one right answer to finding the tableView in the view hierarchy.

Another method is to explicitly search for the accessibilityIdentifier:

let tableView = app.tables.containing(.table, identifier: "MyTable")

If we hadn’t assigned an accessibilityIdentifier, we could use this code to get the first UITableView within the top-level UIView:

let tableView = app.tables.element(boundBy: 0)

This isn’t as good, because if we should ever add a second UITableView to the screen, the UI Test may break if a new UITableView happens to be retrieved as the first UITableView! This is the reason I suggest using accessibilityIdentifiers when designing your UI Tests.

If we knew there were one and only one UITableView on the screen, we could shorten the previous technique even more:

let tableView = app.tables

Again, this has the risk of breaking the UI Test if a second UITableView is added. This would be a more serious break, since the tables property would return a collection rather than a single table as it does when only one UITableView is in the view hierarchy.

Final Test Script

We’ve covered the fundamentals of creating tests, accessing elements, and manipulating values (which Xcode showed us during the test recording), so we’re ready to wrap this up.

I’ve pasted below the final test function, and then annotated it below.

01: func testChangeTableRowText() {
02:     let app = XCUIApplication()
03:     let tableView = app.tables["MyTable"]
04:     XCTAssert(tableView.cells.count == 5)
05: 
06:     let cell = tableView.cells.containing(.cell, identifier: "3")
07:     let cellLabelText = cell.staticTexts.element(boundBy: 0).label
08:     XCTAssertEqual(cellLabelText, "Fourth Row")
09:     
10:     cell.staticTexts.element(boundBy: 0).tap()
11: 
12:     // The detail form is now visible
13:     
14:     XCTAssertEqual(app.staticTexts["labelIdentifier"].label, cellLabelText)
15:     
16:     let textField = app.otherElements.textFields["newValue"]
17:     textField.tap()
18:     textField.typeText("Some new value")
19: 
20:     XCTAssertEqual(textField.value as? String ?? "", "Some new value")
21:     
22:     app.navigationBars["UITestingDemo.DetailView"].buttons["Back"].tap()
23: 
24:     // The detail form is now visible
25: 
26:     let tableView2 = app.tables.containing(.table, identifier: "MyTable")
27:     let cell2 = tableView2.cells.containing(.cell, identifier: "3")
28:     let updatedText = cell2.staticTexts.element(boundBy: 0).label
29: 
30:     XCTAssertEqual(updatedText, "Some new value")
31: }
  • In lines 2–4, we find the UITableView with the accessibilityIdentifier “MyTable”, and then check that the number of rows is five (5). Remember that whenever an XCTAssert fails, the entire test fails.
  • On line 6, we search the UITableView for a UITableViewCell with an accessibilityIdentifier equal to “3”. This value was set in the cellForRowAt method in the UITableView DataSource delegate (review the code from GitHub for details)
  • On line 7, we get the first UILabel within the cell (this cell has only one label).
  • One line 8, the UILabel text property is checked against an expected value (this is not really a requirement for this test, but I added it as a further example).
  • Line 10 sends a tap event to the UILabel within the cell. The effect of this is to generate a tap event on the cell, which then triggers a segue to the detail form (see source on GitHub for details)
  • Line 14 finds the UILabel with accessibilityIdentifier “labelIdentifier” (we set this in Interface Builder earlier. When the form is loaded, it should have set the UILabel text to the value tapped in the UITableView. This XCAssetEqual check to make sure this was done.
  • Lines 16–20 tap on the UITextField, and type in new text.
  • Line 22 taps the Back button at the top-left of the detail form, which pops the view controller off the stack, returning to the first form.
  • Lines 26–30 again retrieve the value in the 4th cell, and compare the new value to the value that was typed on the detail form.

Note: when creating tests that type into fields using an iOS simulator, be sure to disconnect the hardware keyboard in the simulator. The typeText method will fail when a hardware keyboard is attached to the simulator.

Where to go Next

With this, we’ve completed a completed, robust UI Test for this part of the application!

Since we used accessibilityIdentifier properties wherever possible, we’ve created a test that won’t easily break when the UI is enhanced with new controls, and the test is repeatable, automate-able, and easy to use for regression testing.

But this test can be improved even more:

  • We still have a few static data values in the test, e.g. “Fourth Row”. By refactoring all static value assumptions out of this test, we could set it up to work with dynamic data (for example, against a web service call)
  • This test is still bound to a developer or QA Engineer using Xcode at their desk. But with some additional work, we could incorporate this type of test into a fully automated test suite run by a daemon instead. Look for that in a future blog post!

Leave a Reply

Your email address will not be published. Required fields are marked *