Cloud computing is now the foundation for digital transformation. Starting as a technology disruptor a few years back, it has become the de facto approach for technology transformation initiatives. However, many organizations still struggle to optimize cloud adoption. Reasons abound – ranging from lack of a cohesive cloud strategy to mindset challenges in adopting cloud platforms. Irrespective of the reason, assuring the quality of applications in cloud environments remains a prominent cause for concern.
Studies indicate a wastage of $17.6 billion in cloud spend in 2020 due to multiple factors like idle resources, overprovisioning, and orphaned volumes and snapshots (Source parkmycloud.com). Further, some studies have pegged the cost of software bugs to be 1.1 trillion dollars. Assuring the quality of any application hosted on the cloud not only addresses its functional validation but also its performance-related aspects like load testing, stress testing, capacity planning, etc invariably addressing both the issues described above, thereby exponentially reducing the quantum of loss incurred on account of poor quality.
The complication for QA in cloud-based application arises due to many deployment models ranging from private cloud, public cloud to hybrid cloud, and application service models ranging from IaaS, PaaS, to SaaS. While looking at deployment models, testers will need to address infrastructure aspects and application quality. At the same time, while paying attention to service models, QA will need to focus on the team’s responsibilities regarding what they own, manage, and delegate.
Key aspects that mandate a shift in the QA approach in cloud-based environments are –
Application architecture
Earlier and to some extent even now, when it comes to legacy applications, QA primarily deals with a monolithic architecture. The onus was on understanding the functionality of the application and each component that made up the application, i.e., QA was not just black-box testing. The emergence of the cloud brought with it a shift to microservices architecture, which completely changed testing rules.
Multiple scrum teams work on various application components or modules deployed in containers and connected through APIs in a microservices-based application. The containers have a communication mechanism based on contracts. QA methodology for cloud-based applications is very different from that adopted for monolith applications and therefore requires detailed understanding.
Security, compliance, and privacy
In typical multi-cloud and hybrid cloud environments, the application is hosted in a 3rd party environment or multiple 3rd party environments. Such environments can also be geographically distributed, with data centers housing the information residing in numerous countries. Regulations that restrict data movement outside countries, service models that call for multi-region deployment, and corresponding data storage and access without impinging on regulatory norms need to be understood by QA personnel.QA practitioners also need to be aware of the data privacy rules existing across regions.
The rise of the cloud has given way to a wide range of cybersecurity issues – techniques for intercepting data and hacking sensitive data. To overcome these, QA teams need to focus on vulnerabilities of the application under test, networks, integration to the ecosystem, and third-party software deployed for complete functionality. Usage of tools to simulate Man In The Middle (MITM) attacks helps QA teams identify and overcome any sources of vulnerability through countermeasures.
Building action-oriented QA dashboards need to extend beyond depicting quality aspects to addressing security, infrastructure, compliance, and privacy.
Scalability and distributed ownership
Monolithic architectures depend on vertical scaling to address increased application loads, while in a cloud setup, this is more horizontal in nature. Needless to say that in a cloud-based architecture, there is no limitation to application scaling. Performance testing in a cloud architecture need not consider aspects like breakpoint testing since they can scale indefinitely.
With SaaS-based models, the QA team needs to be mindful that the organization may own some components that require testing. Other components that require testing may be outsourced to other providers, and some of these providers may include cloud providers. The combination of on-premise components and others on the cloud by the SaaS provider makes QA complicated.
Reliability and Stability
This entirely depends on the needs of the organization. An Amazon that deploys 100,000 times a day – features and updates of its application hosted in cloud vis-a-vis an aircraft manufacturer that ensures the complete update of its application before its aircraft is in the air, have diverse requirements for reliability stability. Ideally, testing done for reliability should uncover four categories – what we are aware of and understand, what we are aware of but do not understand, what we understand but are not aware of, and what we neither understand nor are we aware of.
Initiatives like chaos testing aim to uncover these streams by randomly introducing failures through automated testing and scripting and seeing how the application reacts/sustains in this scenario.
QA needs to address the below in a hybrid cloud setup are –
- What to do when one cloud provider goes down
- How can the load be managed
- What happens to disaster recovery sites
- How does it react when downtime happens
- How to ensure high availability of application
Changes in organization structure
Cloud-based architecture calls for development through pizza teams, smaller teams fed by one or two pizzas, in common parlance. These micro product teams have testing embedded in them, translating into a shift from QA to Quality Engineering (QE). The tester in the team is responsible for engineering quality by building automation scripts earlier in the cycle, managing performance testing strategies, and understanding how things get impacted in a cloud setup. Further, there is also increased adoption of collaboration through virtual teams, leading to a reduction in cross-functional QA teams.
Tool and platform landscape
A rapidly evolving tool landscape is the final hurdle that the QA practitioner must overcome to test a cloud-based application. The challenge becomes orchestrating superior testing strategies by using the right tools and the correct version of tools. Quick learning ability to keep up with this landscape is paramount. An open mindset to adopt the right toolset for the application is needed rather than an approach steeped with blinders towards toolsets prevailing in the organization.
In conclusion, the QA or QE team behaves like an extension of customer organization since it owns the mandate for ensuring the launch of quality products to market. The response times in a cloud-based environment are highly demanding since the launch time for product releases keeps shrinking on account of demands from end customers and competition. QA strategies for cloud-based environments need to keep pace with the rapid evolution and shift in the development mindset.
Further, the periodicity of application updates has also radically changed, from a 6-month upgrade in a monolith application to feature releases that happen daily, if not hourly. This shrinking periodicity translates into an exponential increase in the frequency of test cycles, leading to a shift-left strategy and testing done in earlier stages of the development lifecycle for QA optimization. Upskilling is also now a mandate given that the tester needs to know APIs, containers, and testing strategies that apply to contract-based components compared to pure functionality-based testing techniques.
Wish to know more? Feel free to reach out to us.
