Test iOS Apps with UI Automation Book Review

This book has made creating iOS Automation scripts much easier than it would have been with the Apple Instruments User Guide alone. Although, I do think reading the Apple Instruments User Guide is worthwhile. The Instruments User Guide gives you a nice overview of Instruments, but does not come close to the information this book provides on UI automation. I also recommend checking out the Apple UI Automation JavaScript Reference. At least breeze through it to get an idea of what it contains. The book starts off by jumping right into capturing a script from the simulator and provides a nice overview of the Instrument's trace-document window. Then Chapter 2 jumps right in to creating automated acceptance-test scripts using JavaScript, which is the language used for UI Automation. One thing I must admit is that because of the book's size I really underestimated how valuable the information would be and how many topics would be covered. I normally only list the chapter to give you an idea of what the book covers, but for this review I have also include the sections in each chapter. Chapter 1: UI Automation Overview 1.1 Capturing Our First Script from the Simulator 1.2 Finding Our Way around UI Automation Chapter 2: Testing Behavior with UI Automation 2.1 Talking to the UI through JavaScript 2.2 Reporting Errors 2.3 Verifying that the Test Does What It Says Chapter 3: Building a Test Suite 3.1 Testing with a Modal Alert View 3.2 Importing Script Files at Runtime 3.3 Testing Results from a Live API 3.4 Grouping Test Steps and Their Output Chapter 4: Organizing Test Code 4.1 Starting a JavaScript Toolbox 4.2 Describing the App with Screen Objects 4.3 Reusing a Generic Screen Prototype 4.4 Converting Our Test Suite to Screen Objects Chapter 5: Maps, Gestures, and Alerts 5.1 Exploring the Map 5.2 Identifying Elements with Accessibility APIs 5.3 Testing with Gestures 5.4 Advanced Alert-Handling Chapter 6: Strategies for Testing Universal Apps 6.1 Universalizing an Application 6.2 Finding Elements in the New Idiom 6.3 Building an iPad Test Suite with Reusable Pieces 6.4 Searching the Element Tree with Predicates 6.5 Advanced Predicate Usage and Beyond Chapter 7: Automating Performance Tests 7.1 Setting Up Custom Instruments Templates 7.2 Capturing Steps to Reproduce a Memory Leak 7.3 Triggering Simulator Memory Warnings with AppleScript 7.4 Stress Testing Chapter 8: Setting Up Application Data 8.1 Seeding Data in Xcode with Application Data Packages 8.2 Seeding Data Dynamically with a Factory 8.3 Choose Your Own Adventure with Environment Variables 8.4 Hiding Test-Setup Code from Release Chapter 9: Stubbing External Services 9.1 Choosing a Geographical Location 9.2 Faking a Network-Service API 9.3 Wrapping Service APIs in a Facade 9.4 Stubbing a Facade with Data in the App Bundle Chapter 10: Command-Line Workflow 10.1 Practicing with the Raw Commands 10.2 Automating the Build-and-Run Process with Rake 10.3 Reading Environment Variables from Script Files 10.4 Resetting the Simulator to Test Permissions 10.5 Running Tests on an Attached Device Chapter 11: Third-Party Tools and Beyond 11.1 Enhancing UI Automation 11.2 Testing outside the Box As you can see above the author covers a ton of topics. He also covers them in-depth. There is no filler text in this book. The author also has an excellent writing style which makes reading the text a pleasure. Although the coverage of Instruments and writing scripts is what this book is all about, I must say the real value is the author's understanding of what it takes to do automated testing. If you have no experience with how difficult it can be, I would recommend reading Experiences of Test Automation. The book covers most of the things you must think about when you are including automated testing in your development process. Often times the organization of the testing scripts is overlooked and before you know it you are spending more time fixing your tests than you are writing your code. In the chapter Organizing Test Code, the author does a great job of showing you how to organize your code by building a JavaScript Toolbox. When building universal applications you must build test scripts that account for the iPad and the iPhone interface. The chapter Strategies for Testing Universal Apps shows you how to do that. With iOS performance is always one of the top quality attributes we need to give a lot of attention to. The chapter Automating Performance Tests shows us just how to do that. I also like that the author introduced custom templates in this chapter. A few months ago one of our development teams asked us to write some Load Runner scripts for their application. It is a big COTS application that took 1.5 years to customize and configure, and that was only about 30% of what they had originally planned to do in 1 year. We gave them the list of what they would need to supply, which included a lot of things, but setting up application data and handling external services were included in the list. After they saw the list of what we would require from them for us to implement a test suite they decided not to move forward. The source code examples the author supplies is organized and very usable. The code is broken down by chapters, and then within each chapter each it is broken down into steps. Right now one of the projects I am on has outsourced the development of the project's iOS and Android development. Our team is just creating the services the devices will communicate with. As we get new builds of the iOS app to be tested, it is really nice having the scripts setup to run through the scenarios that should work, and also those that shouldn't work. Running the scripts makes sure nothing unexpected happened, and if it did, we can stop that build from going to the testers and save them a lot of time. All in all this book is a must read for the iOS architect, developer, and tester.

Test iOS Apps with UI Automation: Bug Hunting Made Easy