4 Rs for Scaling Outsourced QA. The first steps towards a rewarding engagement

Expanding nature of products, need for faster releases to market much ahead of competition, knee jerk or ad hoc reactions to newer revenue streams with products, ever increasing role of customer experience across newer channels of interaction, are all driving the need to scale up development and testing. With the increased adoption of DevOps, the need to scale takes a different color altogether.

Outsourcing QA has become the norm on account of its ability to address the scalability of testing initiatives and bring in a sharper focus on outcome-based engagements. The World Quality Report 2020 mentions that 34% of respondents felt QA teams lack skills especially on the AI/ML front. This further reinforces their need to outsource for getting the right mix of skill sets so as to avoid any temporary skill set gaps.

However, ensuring that your outsourced QA gives you speed and scale can be a reality only if the rules of engagement with the partner are clear. Focusing on 4 R’s as outlined below while embarking on the outsourcing journey, will help you derive maximum value.

  1. Right Partner
  2. Right Process
  3. Right Communication
  4. Right Outcome

Right Partner

The foremost step is to identify the right partner, one with a stable track record, depth in QA, domain as well as technology, and the right mix of skill sets across toolsets and frameworks. Further, given the blurring lines between QA and development with testing being integrated across the SDLC, there is a strong need for the partner to have strengths across DevOps, CI/CD in order to make a tangible impact on the delivery cycle.

The ability of the partner to bring to table prebuilt accelerators can go a long way in achieving cost, time and efficiency benefits. The stability or track record of the partner translates to the ability to bring onboard the right team which stays committed throughout the duration of the engagement. The team’s staying power assumes special significance in longer duration engagements wherein shifts in critical talent derails efficiency and timelines on account of challenges involved with newer talent onboarding and effective knowledge transfer.

An often overlooked area is the partner’s integrity. During the evaluation stages, claims pertaining to industry depth as well as technical expertise abound and partners tend to overpromise. Due care needs to be exercised to know if their recommendations are grounded in delivery experience. Closer look at the partner’s references and past engagements not only help to gain insight into their claims but also help to evaluate their ability to deliver in your context.

It’s also worthwhile to explore if the partner is open to differentiated commercial models that are more outcome driven and based on your needs rather than being fixated on the traditional T&M model.

Right Process

With the right partner on board, creating a robust process and governing mechanism assumes tremendous significance. Mapping key touchpoints from the partner side, aligning them to your team, and identifying escalation points serve as a good starting point. With agile and DevOps principles having collaboration across teams as the cornerstone, development, QA, and business stakeholder interactions should form a key component of the process. While cross-functional teams with Dev QA competencies start off each sprint with a planning meeting, formulating cadence calls to assess progress and setting up code drop or hand off criteria between Dev and QA can prevent Agile engagements from degrading into mini waterfall models.

Bringing in automated CI/CD pipelines obviates the need for handoffs substantially. Processes then need to track and manage areas such as quality and release readiness, visibility across all stages of the pipeline through reporting of essential KPIs, documentation for managing version control, resource management, and capacity planning. At times, toolset disparity between various stages and multiple teams driving parallel work streams creates numerous information silos leading to fragmented visibility at the product level. The right process should focus on integration aspects as well to bridge these gaps. Each team needs to be aware and given visibility on ownership at each stage of the pipeline.

Further, a sound process also brings in elements of risk mitigation and impact assessment and ensures adequate controls are built into SOP documents to circumvent any unforeseen event. Security measures is another critical area that needs to be incorporated into the process early on, more often it is an afterthought in the DevOps process. Puppet 2020 State of DevOps report mentions that integrating security fully into the software delivery process can quickly remediate critical vulnerabilities – 45% of organizations with this capability can remediate vulnerabilities within a day.

Right Communication

Clear and effective communication is an integral component of QA, more so when DevOps, Agile, and similar collaboration-heavy initiatives are pursued achieving QA at scale. Effective communication at the beginning of the sprint ensures that cross-functional teams are cognizant of the expectations from each of them and have their eye firmly fixed on the end goal of application release. From then on, a robust feedback loop, one that aims at continuous feedback and response, cutting across all stages of the value chain, plays a vital role in maintaining the health of the DevOps pipeline.

While regular stand-up meetings have their own place in DevOps, effective communication needs to go much beyond to focus on tools, insights across each stage, and collaboration. A wide range of messaging apps like Slack, email, and notification tools accelerate inter-team communication. Many of these toolkits are further integrated with RSS feeds, google drive, and various CI tools like Jenkins, Travis, Bamboo, etc. making build pushes and code change notifications fully automated. Developers need notifications when a build fails, testers need them when a build succeeds and Ops need to be notified at various stages depending on the release workflow.

The toolkits adopted by the partner also need to extend communication to your team. At times, it makes sense for the partner to have customer service and help desk support as an independent channel to accept your concern. The Puppet report further mentions that companies at a high level of DevOps maturity use ticketing systems 16% more than what is used by companies at the lower end of the maturity scale. Communication of the project’s progress and evolution to all concerned stakeholders is integral irrespective of the platforms used. Equally important is the need to categorize communication in terms of priority and based on what is most applicable to classes of users.

Documentation is an important component of communication and from our experiences, commonly underplayed. It is important for sharing work, knowledge transfer, continuous learning and experimentation. Code that is well documented enables faster completion of audit as well. In CI/CD based software release methodology, code documentation plays a strong role in version control across multiple releases. Experts advocate continuous documentation as core communication practice.

Right Outcome

Finally, it goes without saying that setting parameters for measuring the outcome, tracking and monitoring those, determines the success of the partner in scaling your QA initiatives. Metrics like velocity, reliability, reduced application release cycles and ability to ramp up/ramp down are commonly used. Further, there are also a set of metrics aimed at the efficiency of the CI/CD pipeline, like environment provisioning time, features deployment rate, and a series of build, integration, and deployment metrics. However, it is imperative to supplement these with others that are more aligned to customer-centricity – delivering user-ready software faster with minimal errors at scale.

In addition to the metrics that are used to measure and improve various stages of the CI/CD pipeline, we also need to track several non-negotiable improvement measures. Many of these like deployment frequency, error rates at increased load, performance & load balancing, automation coverage of delivery process and recoverability helps to ascertain the efficiency of QA scale up.

Closely following on the heels of an earlier point, an outcome based model which maps financials to your engagement objectives will help to track outcomes to a large extent. While the traditional T&M model is governed by transactional metrics, project overlays abound in cases where engagement scope does not align well to outcome expectations. An outcome based model also pushes the partner to bring in innovation through AI/ML and similar new age technology drivers – providing you access to such skill sets without the need for having them on your rolls.

If you are new to outsourcing, or working with a new partner, it may be good to start with a non-critical aspect of the work (for regular testing or automation), establish the process and then scale the engagement. For those players having maturity in terms of adopting outsourced QA functions in some way or the other, the steps outlined earlier form an all inclusive checklist to ensure maximization of engagement traction and effectiveness with the outsourcing partner.

Partner with us

Trigent’s experienced and versatile Quality Assurance and Testing team is a major contributor to the successful launch, upgrade, and maintenance of quality software used by millions around the globe. Our experienced responsible testing practices put process before convenience to delight stakeholders with an impressive industry rivaled Defect Escape Ratio or DER of 0.2.

Trigent is an early pioneer in IT outsourcing and offshore software development business. We enable organizations to adopt digital processes and customer engagement models to achieve outstanding results and end-user experience. We help clients achieve this through enterprise-wide digital transformation, modernization, and optimization of their IT environment. Our decades of experience, deep domain knowledge, and technology expertise delivers transformational solutions to ISVs, enterprises, and SMBs.

Contact us now.

Poor application performance can be fatal for your enterprise, avoid app degradation with application performance testing

If you’ve ever wondered what can possibly go wrong’ after creating a foolproof app, think again. Democrats’ Iowa Caucus voting app is a case in point. The Iowa caucus post-mortem pointed towards a flawed software development process and insufficient testing.

The enterprise software market is predicted to touch US$230,134.0m in 2021, and the revenue is expected to grow with a CAGR of 9.1% leading to a market volume of US$326,285.5m by 2025. It is important that enterprises aggressively work towards getting their application performance testing efforts on track to ensure that all the individual components that go into the making of the app provide superior responses to ensure a better customer experience.

Banking app outages have also been pretty rampant in recent times putting the spotlight on the importance of application performance testing. Customers of Barclays, Santander, and HSBC suffered immensely when their mobile apps suddenly went down. It’s not as if banks worldwide are not digitally equipped. They dedicate at least 2-3 percent of their revenue to information technology along with additional expenses on building a superior IT infrastructure. What they also need is early and continuous performance testing to address and minimize the occurrence of such issues.

It is important that the application performs well not just when it goes live but later too. We give you a quick lowdown on application performance testing to help you gear up to meet modern-day challenges.

Application performance testing objectives

In general, users today, have little or no tolerance for bugs or poor response times. A faulty code can also lead to serious bottlenecks that can eventually lead to slowdown or downtime. Meanwhile, bottlenecks can arise from CPU utilization, disk usage, operating system limitations, or hardware issues.

Enterprises, therefore, need to conduct performance testing regularly to:

  • Ensure the app performs as expected
  • Identify and eliminate bottlenecks through continuous monitoring
  • Identify & eliminate limitations imposed by certain components
  • Identify and act on the causes of poor performance
  • Minimize implementation risks

Application performance testing parameters

Performance testing is based on various parameters that include load, stress, spike, endurance, volume, and scalability. Resilient apps can withstand increasing workloads, high volumes of data, and sudden or repetitive spikes in users and/or transactions.

As such, performance testing ensures that the app is designed keeping peak operations in mind and all components comprising the app function as a cohesive unit to meet consumer requirements.
No matter how complex the app is, performance testing teams are often required to take the following steps:

  • Setting the performance criteria – Performance benchmarks need to be set and criteria should be identified in order to decide the course of the testing.
  • Adopting a user-centric approach – Every user is different and it is always a good idea to simulate a variety of end-users to imagine diverse scenarios and test for use cases accordingly. You would therefore need to factor in expected usage patterns, the peak times, length of an average session within the application, how many times do users use the application in a day, what is the most commonly used screen for the app, etc.
  • Evaluating the testing environment – It is important to understand the production environment, the tools available for testing, and the hardware, software, and configurations to be used before beginning the testing process. This helps us understand the challenges and plan accordingly.
  • Monitoring for the best user experience – Constant monitoring is an important step in application performance testing. It will give you answers to what, when, and why’ helping you fine-tune the performance of the application. How long does it take for the app to load, how does the latest deployment compare to previous ones, how well does the app perform while backend performances occur, etc. are things you need to assess. It is important that you leverage your performance scripts well with proper correlations, and monitor performance baselines for your database to ensure it can manage fresh data loads without diluting the user experience.
  • Re-engineering and re-testing – The tests can be rerun as required to review and analyze results, and fine-tune again if necessary.

Early Performance Testing

Test early. Why wait for users to complain when you can proactively run tests early in the development lifecycle to check for application readiness and performance? In the current (micro) service-oriented architecture approach, as soon as the component or an interface is built, performance testing at a smaller scale can allow us to uncover issues w.r.t concurrency, response time/latency, SLA, etc. This will allow us to identify bottlenecks early and gain confidence in the product as it is being built.

Performance testing best practices

For the app to perform optimally, you must adopt testing practices that can alleviate performance issues across all stages of the app cycle.

Our top recommendations are as follows:

  • Build a comprehensive performance model – Understand your system’s capacity to be ready for concurrent users, simultaneous requests, response times, system scalability, and user satisfaction. The app load time, for instance, is a critical metric irrespective of the industry you belong to. Mobile app load times can hugely impact consumer choices as highlighted in a study by Akamai which suggested conversion rates reduce by half and bounce rate increases by 6% if a mobile site load time goes up from 1 second to 3. It is therefore important that you factor in the changing needs of customers to build trust, loyalty, and offer a smooth user experience.
  • Update your test suite – The pace of technology is such that new development tools will debut all the time. It is therefore important for application performance testing teams to ensure they sharpen their skills often and are equipped with the latest testing tools and methodologies.

An application may boast of incredible functionality, but without the right application architecture, it won’t impress much. Some of the best brands have suffered heavily due to poor application performance. While Google lost about $2.3 million due to the massive outage that occurred in December 2020, AWS suffered a major outage after Amazon added a small amount of capacity to its Kinesis servers.

So, the next time you decide to put your application performance testing efforts on the back burner, you might as well ask yourself ‘what would be the cost of failure’?

Tide over application performance challenges with Trigent

With decades of experience and a bunch of the finest testing tools, our teams are equipped to help you across the gamut of application performance right from testing to engineering. We test apps for reliability, scalability, and performance while monitoring them continuously with real-time data and analytics.

Allow us to help you lead in the world of apps. Request a demo now.

Improve the quality of digital experiences with Performance Engineering

Quality at the heart of business performance

“In 2020, the key expectation is fast, reliable, and trustworthy software.” *

As businesses embrace the Agile/DevOps culture and the emphasis on CI/CD is growing, quality assurance is seen as a standalone task, limited to validating functionalities implemented. When QA and Testing is an afterthought in an Agile/DevOps culture, the result is a subpar consumer experience followed by an adverse impact on the revenue pipeline. Poor customer experience also directly impacts brand credibility and business equity. While UI/UX are the visible elements of the customer experience, product, or service performance is a critical element that is often neglected. Performance Testing identifies the gaps that are addressed through Performance Engineering.

Small steps, significant gains – the journey towards Performance Engineering

The deeper issue lies in the organization’s approach towards quality and testing – it is considered an independent phase rather than looked upon as a collaborative and an integrated approach. Performance engineering is a set of methodologies that identifies potential risks and bottlenecks early on in the development stage of the product and addresses them. It goes without saying that performance is an essential ingredient in the quality of the product, there’s a deeper need for change in thinking – to think proactively, anticipate early in the development cycle, test and deliver a quality experience to the end consumer. An organization that makes gradual changes in its journey towards performance engineering stands to gain significantly. The leadership team, the product management, and the engineering and DevOps at different levels need to take the shift-left approach towards performance engineering.

Make Performance Engineering your strategic priority today

Despite the obvious advantages, performance testing is typically a reactive measure that is addressed after the initial launch. However, organizations need to embrace performance engineering measures right from the design phase, start small, and take incremental steps towards change.

Covid-19 has rapidly changed the way consumers behave globally. Businesses caught onto remote working; consumers moved shopping, entertainment, banking, learning, and medical consultations online. Consider the quantum jump in usage triggered by the pandemic.

The dramatic increase in the use of digital services has covered decades in days.**

Companies that adopted scalability and performance centric design have moved swiftly to capture the market opportunity.

With multiple user-interfaces across sectors being the norm and the increasing complexity of digital experiences, it is critical for businesses to get it right the first time in order to gain and retain customers’ trust.

As cloud migrations continue, whether rehosting the app on an IaaS or rebuilding a new approach, performance engineering ensures that migrated systems withstand sudden surges in usage. According to a Sogeti and Neotys report, 74% of the load testing infrastructure is operated in the cloud today. Cloud infrastructure providers ensure reliability but they may not be aware of the performance metrics that matter to the business and their impact. As organizations move from monolithic systems to distributed architectures provided by an assortment of companies, corporate leaders need to recognize the importance of performance engineering and embrace it to deliver the right solutions for the first time.

Our approach to Performance Engineering philosophy

At Trigent, we put the customer experience at the heart of planning the entire testing cycle. Our performance engineering practices align with ‘metrics that matter’ to businesses in the DevOps framework. While testing identifies the gaps in performance, the onus of architecting it right lies on the DevOps engineering team with proactive inputs from QA and Testing.

Performance engineering is also a way of thinking, the ability to plan for performance at the time of design, right at the beginning. As for quality, besides testing for functionality, anticipating potential bottlenecks helps us assess the process in its entirety in the beginning.

Asking some of these customer-centric questions early on shifts the perspective right at the outset. Ask them early, and you’re on your way to a performance engineering culture.

Parameters that matter

‘Will my application meet the defined response-time requirements of my customers?’

Consider an app that doesn’t respond within the expected standards of the customer; the chances of that application making it to the customer’s phone screen is pretty slim.

‘Will the application handle the expected user load and beyond?’

An application that tested well with 10 users may fail when that number is multiplied by a thousand or two.

We take the viewpoints of multiple stakeholders, consider parameters that matter to the customer, and assess impact early on.

Customer experience matters

Performance Engineering takes into account the overall experience of the end-user and their environment.

Asking pertinent questions such as ‘Will my users experience acceptable response times, even during peak hours?’ or ‘Does the application respond quickly enough for the intended users?’ does well to anticipate potential pitfalls in network usage and latency.

‘Where are the bottlenecks in my multi-user environment?’

Understand the real environment of the user and their challenges to provide a quality user experience.

Early Focus

The non-functional aspects are integrated into the DevOps and an early focus on performance enables us to gain insights into architectural issues.

‘How can we optimize the multi-user application before it goes live?
‘How can we detect errors that only occur under real-load conditions?

Quick course corrections help optimize performance and make the product market-ready. Besides faster deployment, quality assurance gives our clients an added advantage of reduced performance costs.

Architect it right

‘What system capacity is required to handle the expected load?’
‘Will the application handle the number of transactions required by the business?’

Important questions like these focus on architecting the product for performance. As part of the performance engineering methodology, our teams consistently check and validate the capabilities at the time of developing the product or scaling it up. We take the shift-left and shift-right approach to anticipate, identify, and remove bottlenecks early on. Getting the architecture right enables us to deliver and deploy a high-quality product every time.

Performance engineering done right is sure to improve the planning-to-deployment time with high-quality products. Plus, it reduces performance costs arising out of unforeseen issues. A step-by-step approach in testing makes sure organizations move towards achieving performance engineering. Talk to our experts for scalable performance engineering solutions for your business.

Learn more about Trigent software testing services.


Reference:
* The State of Performance Engineering 2020 – A Sogeti and Neotys report
** Meet the next-normal consumer – A McKinsey & Company report

Five Business Benefits of On-Demand Testing

Constantly shifting economic conditions has resulted in businesses tightening their IT budgets to control costs and remain competitive. However, while budgets are limited, expectations from IT managers are only growing larger. Most companies today have an online presence, and they need to frequently upgrade and innovate their offerings to stay competitive. This has led to new features, apps, and products being unleashed at a rapid space. There is also an uncompromising world of usability that requires released products and apps to be fault-free. However, the bottom line is, budgets remain controlled.

On-demand testing or Testing as a Service (TaaS) is a realistic option for stringent budgets and tight deadlines. Reliable service providers, who offer on-demand testing, have the capabilities for testing in cloud-based or on-premise environments. More often than not, these providers have a wide array of tools, assets, frameworks, and test environments.

End-to-end testing services that you need on on-demand basis.

On-demand testing is offered by companies that are confident of taking on the responsibility of transferred ownership. For QA and IT managers, the risks attached to testing and the costs for tools can be assigned to service providers thereby immediately diminishing both risk and added expenditures.

Unlike other services, on-demand testing is demanding in its expectations. Those offering this service cannot afford to escape, bugs, slipped deadlines, and, therefore, the advantages far outweigh the effort involved in sourcing the right partner.

Some of the benefits of On-Demand Testing are:

1. As costs are negotiated and finalized with the partner, the probability of unexpected expenses is brought down considerably. There is a commitment agreed upon and that helps in better budget allocation. There are instances where clients have experienced over 50 percent savings in costs. However, the savings are dependent on the choice of the service provider.

2. Dynamic and scalable testing requirements benefit from on-demand testing, where the partner’s team can depute several test engineers on to the job, reducing testing time drastically. For example, imagine a small number of 3-5 in house engineers who are caught up in multiple projects, to a dedicated team of any number of test engineers focused on a single project. The advantage in terms of time spent can itself become a key advantage.

3. On-demand testing is especially useful for load, performance, and last-mile testing where real-life scenarios need to be created. With real-life test environments at their disposal, service providers are capable of maximizing test coverage and results in limited time frames.

4. Time-to-market can be reduced when partners are involved, as they will help to plan and schedule test cycles without any delays. In such cases, the saved time can be at least 30 percent.

5. Reliable on-demand testing service providers offer standardized infrastructure, frameworks, and pre-configured environments to ensure that configuration errors do not creep in after release.

Trigent’s SwifTest is a pay-as-you-go software testing service that is best suited to gain instant access to qualified professional software testers without any long-term contracts. Trigent will perform end-to-end functional testing of your web or mobile applications. Our service is offered on inclusive environments of mobile devices, operating systems, and browsers – to help you validate your product at a pace faster than traditional outsourced testing service. Our certified testing specialists will ensure all user functions (links, menus, buttons, etc.) are working properly on target devices and browsers. We’ll perform exploratory testing and follow your specified test steps. This Development-QA follow-the-sun model reduces project duration and increases responsiveness. You pay only for the time that you engage our QA engineers/team-as little as one day (8 hours) / a week / or a month.
Read More