Category Archives: Appium

Continuous Integration and automation for mobile apps

Free Download eBOOK (pay if you find useful)

cover_1

https://leanpub.com/ci-mobile-app

CodeBase

https://github.com/machzqcq/ci_android_acceptancetests

Mobile – Can I live without it?

Mobile Phones are very personal devices. Have you ever lost your phone and said to yourself, “Oh well, I’ll find it another time. It’s just not that important.”Surely not. Once you lose your phone, your biggest priority at that moment is to find it or get a replacement. All other priorities go out the window.

Continuous Integration – Mobile Apps

Continuous Integration, deployment and automated testing is a holy grail for software development projects when talking about MVP(minimum viable product) and feedback loop. One of the biggest challenges for mobile apps is being able to setup and tear down sandbox environments to test an end user experience. The dimension of being able to manage a mobile device in addition to the build machine makes the coordination and synchronization interesting and exciting.

Web Application test automation has been around the corner for quite some time now. WebDriver JSON wire protocol is nowadays used to automate both thick and thin clients.Selenium (with selenium grid) is one of the most popular tool/frameworks/library for automating web applications loaded in browsers.

While some might believe that mobile app development is relatively easier than web application development (barring exceptions), continuous integration, deployment and automation for a mobile is n-fold complex than web app test automation (n value varies case-case basis, at least being 5 in my experience). Some of the challenges are

  1. The number of form factors – screen resolution, devices, SDKs
  2. Native Apps vs. web apps vs. hybrid apps
  3. Different worlds – Android, iOS, FirefoxOS, Windows Mobile and so on..
  4. Simulators/Emulators vs. Real Devices
  5. On-premise mobile lab vs. in-cloud labs
  6. Parallel test execution

What this Book is About?

This will help you get started on a CI journey while developing Android apps and automating the feedback loop (testing) on real Android devices. We prefer real devices over emulators, though testing on emulators might get you started quickly.

Who this Book is for?

Product Owner/Business Analyst – We will follow ATDD (acceptance test driven development), hence requirements enter as Gherkin specs. So the role of a PO here is generally collaborate with other roles and ensure that the features stay current and up-to-date. In a mature ATDD+Agile shop, PO will also be the decision maker if the product has to be deployed into production or not – of course, PO has to be provided with information/data in crisp dashboards to help make that decision. Dashboards that correlate project status, build status, testing status etc.

QA/Test Engineer – Collaborate with Product Owner and Developer to accelerate feedback loop and write automation test cases. A Test Engineer balances both the technical and non-technical requirements that come from PO and developer. A great test engineer never settles and accepts the status quo, but continuously questions the practices, technologies and tools used in the entire software development lifecycle. The tools can be during development, build/release, project management, information access etc.

Developer – Source Code Developer or SDET/Automation Engineer writing automation code to test source code. A great automation engineer looks for opportunities to continuously automate the mundane tasks and helps surface the feedback quickly to the team. Choice of automation technologies are not done in silos. Since Test automation is heavily dependent on synchronization processes from code check-in to production releases including the data that passes in the system, it is quintessential to focus on all touch points and think about how data is passed in the system call path. Thinking early about non-functional requirements like performance testing provides great benefits long term. It goes without saying that collaborating with other roles is a defacto requirement.

Build/Deploy Engineer – If you are responsible for maintaining a build/CI environment like Jenkins, nexus and troubleshoot issues that stop the CI pipeline. Collaborating with the above roles is extremely important, because this role might not have all the information to troubleshoot issues that arise when CI pipeline stops.

Expected Value-Adds from this Book

  • Understand how to build mobile app (android) from source code
  • Set up a development or debugging environment for mobile apps
  • Acceptance Test Driven Development
  • Continuous Integration Concepts and application with tools and code
  • Appium mobile automation library
  • Jenkins Continuous Integration Tool
  • Nexus – Dependency management and Application Release tool
  • SonarQube – Code Quality Analyzer
  • Debugging tools in Android

Feedback

Email: pradeep@seleniumframework.com

Advertisements

Selenium, appium and saucelabs

Background:
I have been battling with the mobile test Automation solutions available in the market for quite sometime and spent crazy amounts of time evaluating the pros and cons of both enterprise tools and open source solutions. The competitive analysis of tools available in the market :

https://saucelabs.com/resources/mobile-testing-tools

Sample Project GitHub Repo:

This is my github project that contains a very simple scenario. Please contribute to this project if you can.

https://github.com/machzqcq/appium_page_object

Description:
Cucumber, Ruby, Selenium and Appium code
Working pieces of code that automate a calculator app
Keyword and Page-object design patterns

Appium integrates with Sauce Labs very easily, in fact appium project is driven by Sauce labs.

Resources and links:
http://appium.io

https://github.com/machzqcq/appium_page_object

https://saucelabs.com

Test Automation Capability Analysis – Ruby Support

Background:

Libraries in programming language offer various capabilities. Jar in Java is equivalent to gem in Ruby. Often when people choose Ruby for Test Automation, there are open questions if Ruby can address the Test Automation needs/capabilities. Below is a list of capabilities that most Test Automation frameworks require. All of the capabilities have been done in Java for years, however due to the benefits of Ruby (growth phase, both script and oops, readable code, meta programming etc.), companies are adopting Ruby in many instances, especially for the Testing function.

The below capability matrix gives us a starting point to talk, does NOT necessarily mean that it is exhaustive and answers all questions. What I have experienced over years is the below matrix gives confidence to move forward with Ruby. I strongly advocate JRuby over MRI/Yarv if your environment already uses Java, because there is lots to leverage from Java World and JRuby helps you leverage/bride with Java world too. As always, there are no absolute answers for all problems (one size fits all) , however risk is minimized with JRuby than MRI/Yarv

Capability Matrix:

S.No Functionality/Business value Gem Name Reference
1 Gherkin/ ATDD English like language Cucumber Reference
2 Web Browser Automation/ layer over selenium Watir-webdriver Reference
3 Web Browser Automation (the real one) / saucelabs selenium-webdriver/saucelabs Reference/Reference
4 Mainframe Automation mainframe-client (te3270-jruby) Reference
5 Mobile apps automation/ Object based identification appium/frank/Calabash/Selenium/cloud platforms Reference
6 Manage gem versions and dependencies bundler Reference
7 Ruby make / Define tasks much like Ant Rake Reference
8 Xml/html parser with xpath or css Nokogiri Reference
9 HTML Reports / Test Execution Reports cucumber reports/dynamic_reports Reference
10 Json/xml / surrogate services json/multi_json/pinch_hitter Reference
11 REST web service calls rest-client/Net-http-client Reference
12 SOAP web service handler savon/Net-http-client Reference
13 Parallel execution of tests on Selenium GRID parallel-tests Reference
14 Fake smtp email server to test emails rumbster Reference
15 Page object model design pattern <define your own> Reference
16 create cucumber project structure and directories Download template or create with hand Reference
17 Test Data YAML In built with Ruby
18 Randomize test data faker Reference
19 Debugging in RubyMine debase Reference
20 Logger for creating log files log4jruby/logging Reference
21 Access oracle database ruby-oci8 Reference
22 Run your own Gem server Multiple options – open the reference link Reference
23 Maintain your own gem repository Geminabox Reference

Pattern for running multiple Cucumber projects on CI server

Background:

One of the most common folder structures of Cucumber projects is as below

Cucumber project stucture
Cucumber project stucture

 

However, often while working with large companies, the projects consist of multiple teams and each team would like to work on their own cucumber projects while “in-sprint”, however at some point they would like to integrate and run all the acceptance-tests (automation scripts) projects on CI server.

As an example, there could be multiple projects each having the structure as above.

How do we ensure that individual teams continue with their work as usual adding code, at the same time have flexibility to run projects (by picking and choosing) at run time. One of the solutions is to have a top level Rakefile where we define tasks to be able to execute features by looping through the projects. The parameters generally for such a task would be:

  1.  tag                 #Cucumber tag to run eg. @smoke tests, @slow tests, @fast tests and so on.
  2. target         # where do you want to run the tests ? locally OR on a Selenium grid
  3. environment # which environment to run eg. integration, sandbox, qa , pre-production etc
  4. browser   # choice of browser viz. chrome, firefox, ie and so on
  5. project(s)  # select which project features to run

At the top folder the following commands will achieve what we want

  1. rake smoke[local,int,chrome] => runs the features with @smoke tag
  2. rake smoke[local,int,chrome,”appium_page_object|calabash_keyword_driven “]=> runs the features with @smoke tag on local, on int environment and on chrome browser on the projects “appium_page_object” and “calabash_keyword_driven”

Rakefile Example:


require 'rubygems'
require 'cucumber'
require 'cucumber/rake/task'
require 'rake'

# rake smoke[local,int,chrome] => runs the features with @smoke tag # on local , on int environment and on chrome browser

# rake #smoke[local,int,chrome,"appium_page_object|calabash_keyword_dri#ven "] => runs the features with @smoke tag on local, on int #environment and on chrome browser
# on the projects "appium_page_object" and #"calabash_keyword_driven"

# I have three projects each running appium, Calabash and Frank.

@projects = %w[appium_page_object calabash_keyword_driven Frank_data_driven]

def taskbuilder

@projects.each do |project_name|
Cucumber::Rake::Task.new(project_name.to_sym) do |t|
t.cucumber_opts = "TARGET=#{ENV['target']} BROWSER=#{ENV['browser']} --tags #{ENV['tags']} "
t.profile = ENV['env']

end
end

end

 

# The task below runs all features with @smoke tag

desc "Run all features with the @smoke tag"
task :smoke, [:target, :env, :browser, :projects] do |t, args|
execute_tests('@smoke',args[:target],args[:env],args[:browser], args[:projects])
end

 

# This method builds and runs the rake tasks with specified tag, #target, environment and browser

def execute_tests(tag,target,env,browser,projects={})

ENV['tags'] = tag
ENV['target'] = target
ENV['env'] = env
ENV['browser'] = browser

results = []

#If projects are specified , then the global array @projects is reset #with the values supplied by user, otherwise it remains

if(not projects.nil? and not projects.empty?)
@projects.clear
@projects = projects.split "|"
end

# Build the cucumber tasks
taskbuilder

@projects.each do |project_name|
Dir.chdir(project_name)
results << collate_results(project_name)
Dir.chdir(File.expand_path(File.dirname(__FILE__)))
end
is_failure results
end

# This method reports any failures

def is_failure(results)
results.each do |result|

unless result
puts "Scripts had failures"
raise BuildFailure
end
end
end
# This method invokes the cucumber rake task for the specified #project_name

def collate_results(project_name)
begin
Rake::Task[project_name].invoke
rescue Exception => e
return false
end
true
end

 

Mobile web browser Automation

Background: There is often a need in many companies to test web applications on mobile web browser for eg. launch safari browser on an iphone, hit the url and test the application (It might be for functional testing or Responsive web design testing). This article talks about function/automation testing on mobile web browser and how Selenium + Appium lets us achieve the same by being able to re-use the selenium scripts written for browsers on a PC/MAC et al.

Instant Karma: Think of Appium as a Selenium GRID on the mobile test automation side. Selenium GRID works using the principle of JSON wire protocol, where the selenium scripts send GET/POST requests to the Selenium GRID (hub and nodes) and those get converted into commands through webdrivers that ultimately do the set,get, click operations and so on. Just like how hub and node needs to be started to be able to use selenium GRID, on the appium side, you will just need to start the Appium server (On Mac, just download the appium.dmg file , extract and run it — similar on the windows side too)

Step1: Run Appium server and lets say it is listening on 127.0.0.1:4724

Appium 1.0 Started
Appium 1.0

Step2:

Copy the below code into Eclipse or any IDE and run it as Java Application. When run on Mac, you should see the iOS/iphone/ipad simulator pop up, open safari browser and hit the url (Assuming that you have Xcode, command line tools installed)

Java Code:


package test

import java.net.MalformedURLException;

import java.net.URL;

import org.openqa.selenium.By;

import org.openqa.selenium.WebElement;

import org.openqa.selenium.remote.DesiredCapabilities;

import org.openqa.selenium.remote.RemoteWebDriver;

import org.testng.Assert;

public class MobileWebTest {

public static void main(String[] args) throws MalformedURLException {

DesiredCapabilities desiredCapabilities = new DesiredCapabilities();

desiredCapabilities.setCapability("browserName", "Safari");

URL url = new URL("http://127.0.0.1:4724/wd/hub");

RemoteWebDriver remoteWebDriver = new RemoteWebDriver(url, desiredCapabilities);

// Navigate to the page print html source

remoteWebDriver.get("http://www.google.com");

System.out.println(remoteWebDriver.getPageSource());

//close the app.

remoteWebDriver.quit();

}

}

Eclipse Java Code
Eclipse Java Code

Ruby Code (should have selenium-webdriver gem installed):


## test.rb file. Run it as "ruby test.rb" or run from an IDE like ##RubyMine/Eclipse

require 'selenium-webdriver'

caps = Selenium::WebDriver::Remote::Capabilities.safari
driver = Selenium::WebDriver.for(:remote, :url => "http://127.0.0.1:4724/wd/hub",:desired_capabilities => caps)

driver.get('http://www.google.com')

puts driver.page_source

driver.quit

RubyMineIDE
RubyMineIDE

Closing Thoughts:

I had a large framework built for a company that uses Selenium Grid for parallel/cross browser testing that uses cucumber, ruby. When the company decided to go mobile and started investing in mobile apps, the first effort was to launch the same application in a mobile browser. All that I had to change was the above start up code in hooks.rb file [If you know the standard layout of a cucumber project, the driver is instantiated in hooks.rb].

For Java based projects, if using TestNG, you would have to change the code where driver is instantiated as per above [@before annotation in TestNG]

Note: The same works on Windows with Android too. Just that the AVD and Android SDK needs to be installed correctly and started

iOS apps Automation with Appium

Jargon to be aware of

  • Xcode – The IDE for developing iOS applications
  • Command-line tools – The set of libraries that are installed on Mac (generally come associated with Xcode)
  • iOS SDK 
  • Apple Instruments
  • UIAutomation -js interface for interacting with elements on iOS apps
  • Apple OSX – The operating system on Mac  (As of writing this post latest is 10.9.2 called Mavericks)
  • I also read this blog which was well written too

Steps

1) On Mac, download the appium.dmg from – http://appium.io/

2) Double click and install it to /Applications – Appium.app should be available now

3) Open the Appium.app and it should open an interface – At this point just click launch and it should start fine with ”

GET /wd/hub/status 200 2ms – 144b” shown in the console

Experiment and dabble around the appium.app for a while. Point to a .app file downloaded from internet, hit the “i” button , which will start the inspector , so that we can view the locator information. Select forcedevice and launch it on multiple device simulators. Get comfortable around it.

4) Install rvm on mac – https://rvm.io/ (Mac comes with ruby, however I worked with 2.1.1)

  • curl -sSL https://get.rvm.io --ruby=2.1.1
  • I would suggest to learn to use rvm and read the sections “RVM”, “Rubies”, “gemsets” in the documentation index on rvm.io website

5) Git clone “https://github.com/appium/appium&#8221; (You can download the zip if you don’t have git installed)

6) Assuming the folder “appium” is present in /Users/<username>/appium

7) Lets launch Xcode.app (The version which gets installed on Osx 10.9.2 is Xcode V5.1 (5B130a) ) — I had lots of issues with this Xcode 5.1 and there are enough # of forums which had similar complaints – It is an apple issue and not much to do with appium. I spent numerous hours debugging appium, but realized that it was Xcode version + Apple Instruments which was the root cause.

8) So I downloaded Xcode V5.0.2 and installed to another location [We cannot have the Xcode.app in /Applications because we already had version 5.1]

9) Install Command-line tools for both versions of Xcode (Download from apple website by searching for command-line tools and installing the compatible versions for the Xcode versions) – I know its painful, but it helps to go through the pain

10) Xcode – preferences – Downloads [Check for  and install updates automatically option] and click the button “Check and install now”

11) Now we need to let osx know to use Xcode 5.0.2

12) xcode-select –switch <path to Xcode.app>

13) For information, run “xcodebuild -showsdks” –> shows the list of  all sdk’s for the corresponding Xcode [We can identify which version of Xcode is being used by looking at the list of sdks]

14 ) Now navigate to /Users/<username>/appium folder

15) cd to sample-code/apps/TestApp

16) Run xcodebuild -sdk iphonesimulator6.1

17) It should run a bunch of steps and build should be successful – the Test.app is put in ‘sample-code/apps/TestApp/build/Release-iphonesimulator’ [You can run Step 16 with different sdks based on requirement]

18) Navigate to ‘/Users/<useranme>/appium/sample-code/examples/ruby/cucumber’ and run the following command

— ‘cucumber features’

At this point, the app should pop up and run the feature [assuming appium is up and running]

Seems easy right.

It is if you are lucky. At least I was working on OSX 10.9.2 and instruments/UIAutomation is extremely flaky. Below are some FAQ if that helps

FAQ

These below errors are for the code downloaded from – https://github.com/appium/appium

1) I get error “cannot call private method…..blah blah”

Ans: Open env.rb file and moved the code under “def desired_capabilities end” –> inside the method “def selenium…..”

2) If the keyboard doesn’t come up in the simulator while running the script, the following line helped me [for some reason, I just had to execute it once and then it started working fine, so removed it later]

def selenium
@driver ||= Selenium::WebDriver.for(:remote, :desired_capabilities => desired_caps, :url => server_url)
@driver.execute_script(“mobile: hideKeyboard”)
end

3) Sometimes my script runs and sometimes it doesn’t

Ans: Yes I experience that on osx 10.9.2 and it is very flaky. There are lot of complaints in the forums and here is one – https://groups.google.com/forum/#!topic/appium-discuss/hWz0DqAqhXw

I would suggest that you get Xcode 5.0 or earlier and the associated command line tools and work with that. That will reduce lot of permutations and combinations of failures — which I went through unfortunately.