Continuous Integration and automation for mobile apps

Free Download eBOOK (pay if you find useful)

https://leanpub.com/ci-mobile-app

CodeBase

https://github.com/machzqcq/ci_android_acceptancetests

Mobile – Can I live without it?

Mobile Phones are very personal devices. Have you ever lost your phone and said to yourself, “Oh well, I’ll find it another time. It’s just not that important.”Surely not. Once you lose your phone, your biggest priority at that moment is to find it or get a replacement. All other priorities go out the window.

Continuous Integration – Mobile Apps

Continuous Integration, deployment and automated testing is a holy grail for software development projects when talking about MVP(minimum viable product) and feedback loop. One of the biggest challenges for mobile apps is being able to setup and tear down sandbox environments to test an end user experience. The dimension of being able to manage a mobile device in addition to the build machine makes the coordination and synchronization interesting and exciting.

Web Application test automation has been around the corner for quite some time now. WebDriver JSON wire protocol is nowadays used to automate both thick and thin clients.Selenium (with selenium grid) is one of the most popular tool/frameworks/library for automating web applications loaded in browsers.

While some might believe that mobile app development is relatively easier than web application development (barring exceptions), continuous integration, deployment and automation for a mobile is n-fold complex than web app test automation (n value varies case-case basis, at least being 5 in my experience). Some of the challenges are

1. The number of form factors – screen resolution, devices, SDKs
2. Native Apps vs. web apps vs. hybrid apps
3. Different worlds – Android, iOS, FirefoxOS, Windows Mobile and so on..
4. Simulators/Emulators vs. Real Devices
5. On-premise mobile lab vs. in-cloud labs
6. Parallel test execution

What this Book is About?

This will help you get started on a CI journey while developing Android apps and automating the feedback loop (testing) on real Android devices. We prefer real devices over emulators, though testing on emulators might get you started quickly.

Who this Book is for?

Product Owner/Business Analyst – We will follow ATDD (acceptance test driven development), hence requirements enter as Gherkin specs. So the role of a PO here is generally collaborate with other roles and ensure that the features stay current and up-to-date. In a mature ATDD+Agile shop, PO will also be the decision maker if the product has to be deployed into production or not – of course, PO has to be provided with information/data in crisp dashboards to help make that decision. Dashboards that correlate project status, build status, testing status etc.

QA/Test Engineer – Collaborate with Product Owner and Developer to accelerate feedback loop and write automation test cases. A Test Engineer balances both the technical and non-technical requirements that come from PO and developer. A great test engineer never settles and accepts the status quo, but continuously questions the practices, technologies and tools used in the entire software development lifecycle. The tools can be during development, build/release, project management, information access etc.

Developer – Source Code Developer or SDET/Automation Engineer writing automation code to test source code. A great automation engineer looks for opportunities to continuously automate the mundane tasks and helps surface the feedback quickly to the team. Choice of automation technologies are not done in silos. Since Test automation is heavily dependent on synchronization processes from code check-in to production releases including the data that passes in the system, it is quintessential to focus on all touch points and think about how data is passed in the system call path. Thinking early about non-functional requirements like performance testing provides great benefits long term. It goes without saying that collaborating with other roles is a defacto requirement.

Build/Deploy Engineer – If you are responsible for maintaining a build/CI environment like Jenkins, nexus and troubleshoot issues that stop the CI pipeline. Collaborating with the above roles is extremely important, because this role might not have all the information to troubleshoot issues that arise when CI pipeline stops.

Expected Value-Adds from this Book

• Understand how to build mobile app (android) from source code
• Set up a development or debugging environment for mobile apps
• Acceptance Test Driven Development
• Continuous Integration Concepts and application with tools and code
• Appium mobile automation library
• Jenkins Continuous Integration Tool
• Nexus – Dependency management and Application Release tool
• SonarQube – Code Quality Analyzer
• Debugging tools in Android

Feedback

Email: pradeep@seleniumframework.com

Test Automation Capability Analysis – Ruby Support

Background:

Libraries in programming language offer various capabilities. Jar in Java is equivalent to gem in Ruby. Often when people choose Ruby for Test Automation, there are open questions if Ruby can address the Test Automation needs/capabilities. Below is a list of capabilities that most Test Automation frameworks require. All of the capabilities have been done in Java for years, however due to the benefits of Ruby (growth phase, both script and oops, readable code, meta programming etc.), companies are adopting Ruby in many instances, especially for the Testing function.

The below capability matrix gives us a starting point to talk, does NOT necessarily mean that it is exhaustive and answers all questions. What I have experienced over years is the below matrix gives confidence to move forward with Ruby. I strongly advocate JRuby over MRI/Yarv if your environment already uses Java, because there is lots to leverage from Java World and JRuby helps you leverage/bride with Java world too. As always, there are no absolute answers for all problems (one size fits all) , however risk is minimized with JRuby than MRI/Yarv

Capability Matrix:

S.No	Functionality/Business value	Gem Name	Reference
1	Gherkin/ ATDD English like language	Cucumber	Reference
2	Web Browser Automation/ layer over selenium	Watir-webdriver	Reference
3	Web Browser Automation (the real one) / saucelabs	selenium-webdriver/saucelabs	Reference/Reference
4	Mainframe Automation	mainframe-client (te3270-jruby)	Reference
5	Mobile apps automation/ Object based identification	appium/frank/Calabash/Selenium/cloud platforms	Reference
6	Manage gem versions and dependencies	bundler	Reference
7	Ruby make / Define tasks much like Ant	Rake	Reference
8	Xml/html parser with xpath or css	Nokogiri	Reference
9	HTML Reports / Test Execution Reports	cucumber reports/dynamic_reports	Reference
10	Json/xml / surrogate services	json/multi_json/pinch_hitter	Reference
11	REST web service calls	rest-client/Net-http-client	Reference
12	SOAP web service handler	savon/Net-http-client	Reference
13	Parallel execution of tests on Selenium GRID	parallel-tests	Reference
14	Fake smtp email server to test emails	rumbster	Reference
15	Page object model design pattern	<define your own>	Reference
16	create cucumber project structure and directories	Download template or create with hand	Reference
17	Test Data	YAML	In built with Ruby
18	Randomize test data	faker	Reference
19	Debugging in RubyMine	debase	Reference
20	Logger for creating log files	log4jruby/logging	Reference
21	Access oracle database	ruby-oci8	Reference
22	Run your own Gem server	Multiple options – open the reference link	Reference
23	Maintain your own gem repository	Geminabox	Reference

Name(required)

Email(required)

Website

Comment(required)

Submit

Δ

Mobile web browser Automation

Background: There is often a need in many companies to test web applications on mobile web browser for eg. launch safari browser on an iphone, hit the url and test the application (It might be for functional testing or Responsive web design testing). This article talks about function/automation testing on mobile web browser and how Selenium + Appium lets us achieve the same by being able to re-use the selenium scripts written for browsers on a PC/MAC et al.

Instant Karma: Think of Appium as a Selenium GRID on the mobile test automation side. Selenium GRID works using the principle of JSON wire protocol, where the selenium scripts send GET/POST requests to the Selenium GRID (hub and nodes) and those get converted into commands through webdrivers that ultimately do the set,get, click operations and so on. Just like how hub and node needs to be started to be able to use selenium GRID, on the appium side, you will just need to start the Appium server (On Mac, just download the appium.dmg file , extract and run it — similar on the windows side too)

Step1: Run Appium server and lets say it is listening on 127.0.0.1:4724

Appium 1.0

Step2:

Copy the below code into Eclipse or any IDE and run it as Java Application. When run on Mac, you should see the iOS/iphone/ipad simulator pop up, open safari browser and hit the url (Assuming that you have Xcode, command line tools installed)

Java Code:

package test
import java.net.MalformedURLException;
import java.net.URL;
import org.openqa.selenium.By;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testng.Assert;
public class MobileWebTest {
public static void main(String[] args) throws MalformedURLException {
DesiredCapabilities desiredCapabilities = new DesiredCapabilities();
desiredCapabilities.setCapability("browserName", "Safari");
URL url = new URL("http://127.0.0.1:4724/wd/hub");
RemoteWebDriver remoteWebDriver = new RemoteWebDriver(url, desiredCapabilities);
// Navigate to the page print html source
remoteWebDriver.get("http://www.google.com");
System.out.println(remoteWebDriver.getPageSource());
//close the app.
remoteWebDriver.quit();
}
}

Eclipse Java Code

Ruby Code (should have selenium-webdriver gem installed):

## test.rb file. Run it as "ruby test.rb" or run from an IDE like ##RubyMine/Eclipse

require 'selenium-webdriver'

caps = Selenium::WebDriver::Remote::Capabilities.safari
driver = Selenium::WebDriver.for(:remote, :url => "http://127.0.0.1:4724/wd/hub",:desired_capabilities => caps)

driver.get('http://www.google.com')

puts driver.page_source

driver.quit

RubyMineIDE

Closing Thoughts:

I had a large framework built for a company that uses Selenium Grid for parallel/cross browser testing that uses cucumber, ruby. When the company decided to go mobile and started investing in mobile apps, the first effort was to launch the same application in a mobile browser. All that I had to change was the above start up code in hooks.rb file [If you know the standard layout of a cucumber project, the driver is instantiated in hooks.rb].

For Java based projects, if using TestNG, you would have to change the code where driver is instantiated as per above [@before annotation in TestNG]

Note: The same works on Windows with Android too. Just that the AVD and Android SDK needs to be installed correctly and started

Name(required)

Email(required)

Website

Comment(required)

Submit

Δ

iOS apps Automation with Appium

Jargon to be aware of

• Xcode – The IDE for developing iOS applications
• Command-line tools – The set of libraries that are installed on Mac (generally come associated with Xcode)
• iOS SDK 
• Apple Instruments
• UIAutomation -js interface for interacting with elements on iOS apps
• Apple OSX – The operating system on Mac  (As of writing this post latest is 10.9.2 called Mavericks)
• I also read this blog which was well written too

Steps

1) On Mac, download the appium.dmg from – http://appium.io/

2) Double click and install it to /Applications – Appium.app should be available now

3) Open the Appium.app and it should open an interface – At this point just click launch and it should start fine with ”

GET /wd/hub/status 200 2ms – 144b” shown in the console

Experiment and dabble around the appium.app for a while. Point to a .app file downloaded from internet, hit the “i” button , which will start the inspector , so that we can view the locator information. Select forcedevice and launch it on multiple device simulators. Get comfortable around it.

4) Install rvm on mac – https://rvm.io/ (Mac comes with ruby, however I worked with 2.1.1)

• curl -sSL https://get.rvm.io --ruby=2.1.1

• I would suggest to learn to use rvm and read the sections “RVM”, “Rubies”, “gemsets” in the documentation index on rvm.io website

5) Git clone “https://github.com/appium/appium” (You can download the zip if you don’t have git installed)

6) Assuming the folder “appium” is present in /Users/<username>/appium

7) Lets launch Xcode.app (The version which gets installed on Osx 10.9.2 is Xcode V5.1 (5B130a) ) — I had lots of issues with this Xcode 5.1 and there are enough # of forums which had similar complaints – It is an apple issue and not much to do with appium. I spent numerous hours debugging appium, but realized that it was Xcode version + Apple Instruments which was the root cause.

8) So I downloaded Xcode V5.0.2 and installed to another location [We cannot have the Xcode.app in /Applications because we already had version 5.1]

9) Install Command-line tools for both versions of Xcode (Download from apple website by searching for command-line tools and installing the compatible versions for the Xcode versions) – I know its painful, but it helps to go through the pain

10) Xcode – preferences – Downloads [Check for  and install updates automatically option] and click the button “Check and install now”

11) Now we need to let osx know to use Xcode 5.0.2

12) xcode-select –switch <path to Xcode.app>

13) For information, run “xcodebuild -showsdks” –> shows the list of  all sdk’s for the corresponding Xcode [We can identify which version of Xcode is being used by looking at the list of sdks]

14 ) Now navigate to /Users/<username>/appium folder

15) cd to sample-code/apps/TestApp

16) Run xcodebuild -sdk iphonesimulator6.1

17) It should run a bunch of steps and build should be successful – the Test.app is put in ‘sample-code/apps/TestApp/build/Release-iphonesimulator’ [You can run Step 16 with different sdks based on requirement]

18) Navigate to ‘/Users/<useranme>/appium/sample-code/examples/ruby/cucumber’ and run the following command

— ‘cucumber features’

At this point, the app should pop up and run the feature [assuming appium is up and running]

Seems easy right.

It is if you are lucky. At least I was working on OSX 10.9.2 and instruments/UIAutomation is extremely flaky. Below are some FAQ if that helps

FAQ

These below errors are for the code downloaded from – https://github.com/appium/appium

1) I get error “cannot call private method…..blah blah”

Ans: Open env.rb file and moved the code under “def desired_capabilities end” –> inside the method “def selenium…..”

2) If the keyboard doesn’t come up in the simulator while running the script, the following line helped me [for some reason, I just had to execute it once and then it started working fine, so removed it later]

def selenium
@driver ||= Selenium::WebDriver.for(:remote, :desired_capabilities => desired_caps, :url => server_url)
@driver.execute_script(“mobile: hideKeyboard”)
end

3) Sometimes my script runs and sometimes it doesn’t

Ans: Yes I experience that on osx 10.9.2 and it is very flaky. There are lot of complaints in the forums and here is one – https://groups.google.com/forum/#!topic/appium-discuss/hWz0DqAqhXw

I would suggest that you get Xcode 5.0 or earlier and the associated command line tools and work with that. That will reduce lot of permutations and combinations of failures — which I went through unfortunately.

Name(required)

Email(required)

Website

Comment(required)

Submit

Δ