Test iOS Apps with UI Automation Book Review
The book starts off by jumping right into capturing a script from the simulator and provides a nice overview of the Instrument's trace-document window.
One thing I must admit is that because of the book's size I really underestimated how valuable the information would be and how many topics would be covered. I normally only list the chapter to give you an idea of what the book covers, but for this review I have also include the sections in each chapter.
Chapter 1: UI Automation Overview
1.1 Capturing Our First Script from the Simulator
1.2 Finding Our Way around UI Automation
Chapter 2: Testing Behavior with UI Automation
2.2 Reporting Errors
2.3 Verifying that the Test Does What It Says
Chapter 3: Building a Test Suite
3.1 Testing with a Modal Alert View
3.2 Importing Script Files at Runtime
3.3 Testing Results from a Live API
3.4 Grouping Test Steps and Their Output
Chapter 4: Organizing Test Code
4.2 Describing the App with Screen Objects
4.3 Reusing a Generic Screen Prototype
4.4 Converting Our Test Suite to Screen Objects
Chapter 5: Maps, Gestures, and Alerts
5.1 Exploring the Map
5.2 Identifying Elements with Accessibility APIs
5.3 Testing with Gestures
5.4 Advanced Alert-Handling
Chapter 6: Strategies for Testing Universal Apps
6.1 Universalizing an Application
6.2 Finding Elements in the New Idiom
6.3 Building an iPad Test Suite with Reusable Pieces
6.4 Searching the Element Tree with Predicates
6.5 Advanced Predicate Usage and Beyond
Chapter 7: Automating Performance Tests
7.1 Setting Up Custom Instruments Templates
7.2 Capturing Steps to Reproduce a Memory Leak
7.3 Triggering Simulator Memory Warnings with AppleScript
7.4 Stress Testing
Chapter 8: Setting Up Application Data
8.1 Seeding Data in Xcode with Application Data Packages
8.2 Seeding Data Dynamically with a Factory
8.3 Choose Your Own Adventure with Environment Variables
8.4 Hiding Test-Setup Code from Release
Chapter 9: Stubbing External Services
9.1 Choosing a Geographical Location
9.2 Faking a Network-Service API
9.3 Wrapping Service APIs in a Facade
9.4 Stubbing a Facade with Data in the App Bundle
Chapter 10: Command-Line Workflow
10.1 Practicing with the Raw Commands
10.2 Automating the Build-and-Run Process with Rake
10.3 Reading Environment Variables from Script Files
10.4 Resetting the Simulator to Test Permissions
10.5 Running Tests on an Attached Device
Chapter 11: Third-Party Tools and Beyond
11.1 Enhancing UI Automation
11.2 Testing outside the Box
As you can see above the author covers a ton of topics. He also covers them in-depth. There is no filler text in this book. The author also has an excellent writing style which makes reading the text a pleasure.
Although the coverage of Instruments and writing scripts is what this book is all about, I must say the real value is the author's understanding of what it takes to do automated testing. If you have no experience with how difficult it can be, I would recommend reading Experiences of Test Automation.
When building universal applications you must build test scripts that account for the iPad and the iPhone interface. The chapter Strategies for Testing Universal Apps shows you how to do that.
With iOS performance is always one of the top quality attributes we need to give a lot of attention to. The chapter Automating Performance Tests shows us just how to do that. I also like that the author introduced custom templates in this chapter.
A few months ago one of our development teams asked us to write some Load Runner scripts for their application. It is a big COTS application that took 1.5 years to customize and configure, and that was only about 30% of what they had originally planned to do in 1 year. We gave them the list of what they would need to supply, which included a lot of things, but setting up application data and handling external services were included in the list. After they saw the list of what we would require from them for us to implement a test suite they decided not to move forward.
The source code examples the author supplies is organized and very usable. The code is broken down by chapters, and then within each chapter each it is broken down into steps.
Right now one of the projects I am on has outsourced the development of the project's iOS and Android development. Our team is just creating the services the devices will communicate with. As we get new builds of the iOS app to be tested, it is really nice having the scripts setup to run through the scenarios that should work, and also those that shouldn't work. Running the scripts makes sure nothing unexpected happened, and if it did, we can stop that build from going to the testers and save them a lot of time.
All in all this book is a must read for the iOS architect, developer, and tester.
Test iOS Apps with UI Automation: Bug Hunting Made Easy