Appium Blog Series 4 – Automated Device Certification Testing On Cloud AWS Device Farm with Appium

In the following blog, I will demonstrate how to conduct a`Device Certification’ testing of an Android app manually on Cloud AWS Device Farm using Appium.

Introduction to AWS Device Farm

AWS Device Farm is an app testing service hosted on cloud infrastructure by Amazon Web Services. It allows users to test and interact with Android, iOS, and Web apps on real, physical phones and tablets of there choice.

Device Farm usage

Running Automated tests of apps with Appium and TestNG framework.

Manually running tests with real time interactions on physical devices hosted remotely on AWS.

Prerequisites to run device certification test’s on AWS Device Farm

  • Java/JDK is installed
  • Eclipse IDE is installed. TestNG is installed.
  • m2e Maven Integration for Eclipse.
  • Apache Maven latest version.
  • AWS Account and IAM User are created.
  • .APK file of android app to be tested.

Preparing Appium Java TestNG Package

  1. To create Appium Java TestNG test suite for a android app please visit my earlier blog How to Automate Testing for Android Native Apps Using Appium
  2. Create a new Maven project in Eclipse of type: maven-archetype-quickstart.
  3. Copy the Appium TestNG class created in step 1 to Maven project under src/test/java.
  4. Above Maven project with all test dependencies should be packaged in a .zip file before uploading to AWS Device Farm, and instructions for this is given in this AWS post Maven Build Process.
  5. Build and package above Maven project using the following command. mvn clean package –DskipTests=true

Maven project structure should look as below, zip-with-dependencies file should contain our test code with all dependencies.

Log in to AWS Device Farm with your IAM user account, and go to AWS Device Farm console under Mobile Services category.

Create a new project, which will serve as a logical work space foryour test runs.

After the above project is created, click on the project and create a new run.

Upload .APK file of your app, and go to the next step.

Upload zip-with-dependencies zip file present in the “target” folder of the Maven project in Eclipse as shown.

Choose devices which are compatible with the app to test.

Specify settings to simulate real-world scenarios and device configurations in the next step.

Click on “Review and start run.” In `Review and start run’ page, click “Confirm and start run”. After test run is complete, test results are shown as below.

Read Other Appium related blogs

How to Automate Testing for Android Native Apps Using Appium

Don’t miss my next blog on – Running Appium Automation Script on Android Emulators

Using Appium automation script to Check SQLite DB Content of a Device!

Appium – A Good Open Source Mobile App Testing Framework

Appium Blog Series – Part 3 – Using Appium automation script to Check SQLite DB Content of a Device!

Appium is an Open Source mobile application user-interface (UI) testing framework, and is used to automate native, hybrid and web mobile apps. It enables cross-platform mobile application testing by using common API for both Android and iOS platform test scripts, thereby enhancing code-reusability.

Appium provides flexibility of writing and running automation tests in any language such as, Java, C#, PHP, Ruby, Python, Perl, Objective-C.

In the following example, we will demonstrate how to connect to Android SQLite database via Appium script using core Java, JDBC, and run SQL Queries to perform data validation checks.

About SQLite database:

SQLite is the default database engine on Android, which is an open source, lightweight transactional database that stores app data to a text file with a file extension .db, on the device. To access this .db file via Appium scripts, we first need to pull the .db file to our local machine. The steps to achieve this are listed below:

  • Download eclipse plugin, com.questiod.sqlitemanager.jar and save in your eclipsedropins folder and restart eclipse.
  • Download latest version of sqlite-jdbc-x.x.x.jar, and add it to java build path in eclipse, by right clicking on project > Properties > Java Build Path > Libraries > Add External JARs >
  • I have a sample sqlite database on my device with name book.db, and it has 1 table Title with 4 records as shown below. Connect Android Device to laptop/computer In USB Debugging mode.
  • In eclipse, click DDMS icon shown below, and select book.db file as shown in below file location on device, click import icon highlighted below to pull this file from device and save it to a folder in pc.

The following code snippet opens a jdbc connection to sqlite database book.db, and run’s select sql query to fetch data from Title table and print results to eclipse console.

Connection con = null;
 Class.forName("org.sqlite.JDBC");
 //create a jdbc connection to book.db located in below file path
 con = DriverManager.getConnection("jdbc:sqlite:E:\test\book.db");
 con.setAutoCommit(false);

Statement sqlstmt = null;
//using above connection, create a statement object
sqlstmt = con.createStatement();
//run select sql query on Title table, and store results in Resultset.
ResultSet res = sqlstmt.executeQuery(“select * from Title;”);
//iterate for all data in Title table and print it.
while (res.next()) {
int BookId = res.getInt(“BookId”);
String BookName = res.getString(“BookName”);
String BookAuthor = res.getString(“BookAuthor”);
System.out.println(“BookId = ” + BookId);
System.out.println(“BookName = ” + BookName);
System.out.println(“BookAuthor = ” + BookAuthor);
System.out.println();
}
res.close();
sqlstmt.close();
con.close();

Read Other Appium related blogs

How to Automate Testing for Android Native Apps Using Appium

Don’t miss my next blog on – Running Appium Automation Script on Android Emulators

Automated Device Certification Testing On Cloud AWS Device Farm with Appium

Appium – A Good Open Source Mobile App Testing Framework

Learn more about Trigent’s automation testing services.

Appium Blog Series – Part 2 – Running Appium Automation Script on Android Emulators

About Appium

Appium is an Open Source mobile application UI testing framework, used to automate native, hybrid and web mobile apps. It enables cross-platform mobile app testing by using common API for both Android and iOS platform test scripts, thereby enhancing code-reusability. Appium provides flexibility of writing and running automation tests in any language such as, Java, C#, PHP, Ruby, Python, Perl, Objective-C.

About Android Emulator

Without an Android device, we can still run automation tests of Android Native App’s using an Emulator which mimics the OS and hardware of an Android device on a computer.

In Android SDK folder, there is an interface called as AVD Manager, which creates a virtual mobile device (Emulator) to run on a computer. Running Emulator starts a virtual console in which we can install Android apps.

In the following example, I will demonstrate automation of Android mobile phone’s default installed calculator app on Android Emulator.

Prerequisites for running Appium automation test for a native Android app on Emulator:

  • Java/JDK is installed and path for JAVA_HOME is set on the windows system.
  • Android SDK is installed and required packages of Android API’s are downloaded in SDK Manager.
  • Intel HAXM is installed.
  • Appium is installed.

Start Android Emulator:

In SDK folder on computer, run AVD Manager.exe, AVD Manager Dialog as shown below:

Click Create button, enter below details to create a new AVD.

AVD Manager shows newly created AVD as below, select this AVD and click Start.

In Launch Options dialog click Launch and in 10 to 15 minutes, Android Emulator will be started as below.

In eclipse create a Java project, with package name Android and class name AndroidEmulatorCalculator.

Set Desired Capabilities:

Following code snippet in AndroidEmulatorCalculator class, sets Desired Capabilities for Calculator App.

// Created object of DesiredCapabilities class.
 DesiredCapabilities capabilities = new DesiredCapabilities();

//Set android deviceName desired capability to Android Emulator.
capabilities.setCapability(“deviceName”, “Android Emulator”);

//Set BROWSER_NAME desired capability to Android.
capabilities.setCapability(“browserName”, “Android”);

//Set android VERSION from device desired capability.
capabilities.setCapability(“platformVersion”, “6.0”);

// Set android platformName desired capability to Android.
capabilities.setCapability(“platformName”, “Android”);

// Set android appPackage desired capability.
capabilities.setCapability(“appPackage”, “com.android.calculator2”);

// Set android appActivity desired capability.
capabilities.setCapability(“appActivity”, “com.android.calculator2.Calculator”);

Android Driver:

Following code snippet Creates an object of AndroidDriver class with Desired Capabilities as mentioned above and sets Appium server address with port number, and launches Android calculator app on device.

AndroidDriver driver;
 driver = new AndroidDriver(new URL("http://127.0.0.1:4723/wd/hub"), capabilities);

The code snippet given below shows a simple addition on calculator app by using id tag to locate calculator app elements.

driver.findElement(By.id("com.android.calculator2:id/digit_4")).click();
 driver.findElement(By.id("com.android.calculator2:id/op_add")).click();
 driver.findElement(By.id("com.android.calculator2:id/digit_8")).click();
 driver.findElement(By.id("com.android.calculator2:id/eq")).click();
 String result = driver.findElement(By.className("android.widget.EditText")).getText();
 System.out.println("Result of Addition is : " + result);

Run Automation Test

Start Appium server, and run AndroidEmulatorCalculator class as TestNG test. Automation test will open calculator app on Emulator, clicks on buttons in this sequence -> 4, +, 8 and =.

Gets result from text area of calculator app, and print it in eclipse console.

Learn more why trigent is one of the best automation testing companies.

Read Other Appium related blogs

How to Automate Testing for Android Native Apps Using Appium

Using Appium automation script to Check SQLite DB Content of a Device!

Automated Device Certification Testing On Cloud AWS Device Farm with Appium

Appium – A Good Open Source Mobile App Testing Framework

Appium Blog Series – Part 1 – How to Automate Testing for Android Native Apps Using Appium

Appium is an open source mobile application user interface (UI) testing framework, and is used to automate native, hybrid and web mobile apps. It enables cross-platform mobile app testing by using common API for both Android and iOS platform test scripts, thereby enhancing code-re-usability. Appium provides flexibility of writing and running automation tests in languages such as, Java, C#, PHP, Ruby, Python, Perl, Objective-C.

Appium Architecture In Android:

Appium is a webserver that exposes a REST API. Appium client establishes connection with Appium Server through JSON Wire Protocol by creating a JSON object called as `Desired Capabilities’. Appium Server then creates an automation session by running a session id for the client, based on Key-Value pairs of Desired Capabilities set in Appium client. Appium Server then connects with the UI-Automator provided by Android SDK. UI-Automator communicates with Bootstrap.jar which will run user commands from the test script, on the mobile app in device.

Prerequisites for running Appium automation test for a native Android app:

  • Java/JDK is installed and path for JAVA_HOME is set on the windows system.
  • Android SDK is installed and required packages of Android API’s are downloaded in SDK Manager. Path for ANDROID_HOME, platform-tools is set on the windows system.
  • Eclipse IDE is installed. TestNG is installed.
  • Eclipse ADT Plugin is installed and configured to integrate android development environment in eclipse IDE.
  • Microsoft .Net Framework and Node JS is Installed.
  • Appium is installed.
  • Latest Selenium Webdriver jar files for java client, Appium Client library jar file, Google-gson jar file are downloaded.
  • Android Device is connected to your laptop/computer In USB Debugging mode.

In eclipse create a Java project, with package name Android and class name AndroidCalculator as shown below.

Add selenium webdriver’s jar files, Appium Client library jar file, Google-gson jar file in to java build path, by right clicking on project > Properties > Java Build Path > Libraries > Add External JARs >

Set Desired Capabilities:

We need to provide the following set of Desired Capabilities to provide Appium Server with details of automation session we want to set up:

  • Android Device Name – In command prompt, run command adb devices to show Android Device Name.
  • Android OS Version – In android device, Open settings -> About phone -> Android version
  • Android Software App Package Name, App Activity Name – In command prompt, run command adb logcat, and open Calculator App in Android phone and press ctrl+c. Log show’s App Package Name = com.android.calculator2, App Activity Name = com.android.calculator2.Calculator.

In class AndroidCalculator, following code snippet, creates a object of DesiredCapabilities class.

DesiredCapabilities capabilities = new DesiredCapabilities();

//Set android deviceName desired capability.
capabilities.setCapability(“deviceName”, “Q88HQ4S4SC89S8NB”);

//Set BROWSER_NAME desired capability to Android.
capabilities.setCapability(CapabilityType.BROWSER_NAME, “Android”);

//Set android VERSION from device desired capability.
capabilities.setCapability(CapabilityType.VERSION, “5.0”);

// Set android platformName desired capability to Android.
capabilities.setCapability(“platformName”, “Android”);

// Set android appPackage desired capability.
capabilities.setCapability(“appPackage”, “com.android.calculator2”);

// Set android appActivity desired capability.
capabilities.setCapability(“appActivity”, “com.android.calculator2.Calculator”);

Android Driver:

Following code snippet creates an object of AndroidDriver class with Desired Capabilities as mentioned above and sets Appium server address with port number, and launches android calculator app on device.

AndroidDriver driver;
 driver = new AndroidDriver(new URL("http://127.0.0.1:4723/wd/hub"), capabilities);

Locate elements in Android App:

To locate elements in Android App, we use UI Automator Viewer tool provided by Android SDK.

  1. Run uiautomatorviewer.bat file from SDKtools folder.
  2. In Android device, open calculator app.
  3. In UI Automator Viewer tool, click on Device Screenshot image button.
  4. UI Automator Viewer now shows calculator app’s screenshot on left pane, and right pane show’s calculator app’s UI element’s hierarchy view with node structure that explain’s how elements are arranged. Ex: In calculator snapshot, button 4 is selected, and right pane shows different properties of button 4, which can be used in test script for locating button 4.

Code The snippet below shows  a simple addition on calculator app by using id tag to locate calculator app elements.

driver.findElement(By.id("com.android.calculator2:id/digit_4")).click();
 driver.findElement(By.id("com.android.calculator2:id/op_add")).click();
 driver.findElement(By.id("com.android.calculator2:id/digit_8")).click();
 driver.findElement(By.id("com.android.calculator2:id/eq")).click();
 String result = driver.findElement(By.className("android.widget.EditText")).getText();
 System.out.println("Result of Addition is : " + result);

Run Automation Test:

Start Appium server, and make configuration changes as shown in the below snapshots and click on Run button.

…to be continued.

Read Other Appium related blogs

Don’t miss my next blog on – Running Appium Automation Script on Android Emulators

Using Appium automation script to Check SQLite DB Content of a Device!

Automated Device Certification Testing On Cloud AWS Device Farm with Appium

Appium – A Good Open Source Mobile App Testing Framework

Why Silk Performer is the Best Performance Testing Tool for Enterprises

What is Silk Performer?

Silk Performer provides advanced facilities for running multi-user tests to validate an application’s performance under different load conditions. It provides on-demand cloud based load generation facility and simulates peak loads from distributed geographies, thereby reducing investment overhead in load-testing hardware setup and maintenance.

Silk Performer provides identification of client-side web page components which can be performance bottlenecks such as image sizes not optimized for performance, client-side resources being called serially wherein an asynchronous implementation may provide web page performance improvements.

  • Silk Performer provides webpage performance breakdown which can show performance bottlenecks of webpage component, client-side network level.
  • Silk Performer provides integration facilities with APM tools AppDynamics and DynaTrace, to resolve performance issues in code such as memory leaks, loitering objects etc.
  • Silk Performer allows creation of custom load test reporting templates to suit business processes and requirements.
  • Silk Performer Performance trend dashboard shows performance changes in the application for frequent build cycles and is useful to identify performance issues early in the development cycle.
  • By adding content verifications in test scripts, it can help identify content errors under load.
  • Silk Performer’s baseline testing approach allows to set acceptable response time SLA’s and subsequently aid, in flagging, transactions whose response time exceeds acceptable response-time.
  • Silk Performer provides load injector health indicators (CPU, Memory, Responsiveness).

Best suited for enterprise-level performance testing services

Silk Performer is a  performance testing and load testing tool that can be considered for fitment in multiple scenarios, as it supports a rich set of industry standard protocols. For e.g. consider the following scenario where checking performance with available software testing tools in the market could be really difficult.

Story/Alert message feeds sent in binary format via a TCP/IP connection with TIBCO enterprise messaging services as Middleware, to a third party NML server which converts these Story/Alert messages to NEWSML format & then forwards this message to BEA Weblogic server cluster, which processes this message and sends it to CMS and issues an output TIBCO broadcast message, to signal that the Story/Alert is successfully sent. These output TIBCO broadcast messages are sent via UDP protocol over TIBCO Enterprise messaging layer. Silkperformer does not support generic recording and playback of UDP based traffic. This limitation was addressed by developing Silk Performer load test scripts in conjunction with a COM DLL interface  which follows event driven architecture, to wait for Tib messages and pulse event when .output message is received.

The objective of performance testing is to Measure the Performance improvements incorporated for NMS and BEA application servers by capturing processing times of each of these layers in above architecture.

(Note: NewsML is a messaging standard in xml format, that is designed to provide a media-independent, structural framework for multi-media news. TIBCO is a set of middleware products)

With Silk Performer’s enhanced TCP/IP support we can record protocols at the TCP level and customize the resulting scripts using WebTcpip functions. In Silk Performer we can create Listener which wait’s for a specific TIBCO message to arrive. The arrival of the TIBCO message is signalled by a Windows Event, pulsed by the StoryListener user.

Silk Performer’s  TCP/IP TrueLog provides visual front-end diagnostics from the end-user perspective. TrueLog visually recreates the data that users provide and receive during full scale load tests which enables us to visually analyze the behavior of an application as errors occur. TrueLog Script Customization helps with a number of activities, such as session correlation. TrueLog automatically detects and customizes session-dependent information that is statically included in test scripts.

Below Silk performer sample code shows:

1.Creation of two user groups – Story_Sender and Story_Listener.

  1. Story_Sender sends a Story/Alert request to NML server, that should be followed by a TIBCO message coming in. Story_Sender starts a timer and is set to wait state, waiting for a specific Window’s event’s state to be set to “signalled”.Once the event is set to signalled, timer is stopped.
  2. Story_Listener is responsible for listening continuously to incoming TIBCO messages. After receiving a new TIBCO message, its message body(containing xml data) is parsed, and a corresponding windows event is signalled. Story_Listener is integrated via COM interface “ISilkPerformerUser” which enables Silkperformer to wait for Tib message’s and pulse event when .output message is received.
 sEvent:= CreateEvent(sEventName);

//Open tcp/ip connection to myServer using port 23

WebTcpipConnect(hWeb,myServer,23);

MeasureStart(Timer1);

MeasureStart(“Timer1 : NML Process Story”);
// Send story message to NML
WebTcpipSendBin(hWeb,sMessage);

WebTcpipShutdown(hWeb);

// The Story_Listener User captures and parses resulting story message to Web Logic. It then pulses “sEvent”

// if successful.
nSuccess:=WaitForSingleObject(sEvent, 20000);
if nSuccess=WAIT_OBJECT_0 then
MeasureStop(“Timer1 : NML Process Story”);

MeasureStart(“Timer2 : WebLogic Processing Time”);
MeasureGet(“Timer1 : NML Process Story”, MEASURE_TIMER_RESPONSETIME, MEASURE_KIND_LAST, fValue);

Print( “Timer1 : NML Process Story took “+ string(fValue)+” seconds”);
// Initialise hEvent to expected .OUTPUT headline

sEventName := “TibMsg:” + sHeadLine;
hEvent := CreateEvent(sEventName);

end;
// Listening for “.OUTPUT” with user TibOutputLstnr. It pulses “hEvent” if successful.
bSuccess:=fWait_Tib(Timer2,sEventName, 40,hEvent);

if (bSuccess) then

MeasureStop(“Timer2 : WebLogic Processing Time”);
MeasureGet(Timer2, MEASURE_TIMER_RESPONSETIME, MEASURE_KIND_LAST, fValue);

Print(” Transaction execution took “+ string(fValue)+” seconds”);

elseif not (bSuccess) then return 0

end;
// fWait_Tib: Wait for a specific TIBCO message to arrive. The arrival of the TIBCO message is signalled by a Windows Event, pulsed by the StoryListener user.
//

// Arguments:

// sTimer: Name of the timer for the time measurement

// sEventName: Name of the Windows event to wait for

// fBound1: Service level (in seconds) for “Green” condition

// fBound2: Service level (in seconds) for “Yellow” condition;

// nTimeout: After waiting for nTimeout seconds, we give up and

// raise an appropriate error message

// Return value:

// True, if the message was received successfully

// False, in case of time out

var

sMsg : string(2000);

sTimestamp: string(100);

begin

if hEvent = 0 then

// Our naming scheme for Windows Events reflecting Tib messages start with “TibMsg:”

sEventName := “TibMsg:” + sEventName;

// Get a handle to a windows event

hEvent := CreateEvent(sEventName);

end;

// Wait for this windows event to be signalled, which will happen when

// the TibListener user receives a Tib messages matching this event

if WaitForSingleObject (hEvent, nTimeout * 2000) = WAIT_OBJECT_0 then

MeasureStop(sTimer);

sMsg := “Received event!”;

fWait_Tib := true;

else

sMsg := “Waiting for event ‘” + sEventName + “‘ timed out after “ + String(nTimeout) + ” seconds”;

RepMessage(“Waiting for event (“ + sEventName + “) timed out after “ + String(nTimeout) + ” seconds”, SEVERITY_ERROR);

fWait_Tib := false;

End;
CloseHandle(hEvent);
end fWait_Tib;

We ran a series of load tests and identified transaction’s queuing up on NML server due to synchronous call’s. After changing to a asynchronous implementation, and rerunning the load tests we were able to achieve peak hour load volumes for Stories and Alerts and were able to get Performance measurements of NML and BEA servers.

SilkPerformer’s inbuilt datastore MSDE was used to archive load test results from multiple test runs. With this data store, we could do a comparison of each test run against baseline run, and build a performance trend across multiple test runs for planned build cycles.

SilkPerformer’s custom reporting templates were useful in preparing test reports providing breakup of response timer’s for NML and BEA servers, and showing 90th and 95th percentile response time achievements of each transaction. This test reports could easily be migrated to MS Excel, MS Word, and  SilkPerformer allows for publishing html version of performance test reports on online dashboards.

Verdict

After using the silk performer in many quality assurance scenarios, we found that it is flexible enough to be used in various scenarios to conduct performance testing, stress testing and load testing.

Exit mobile version