Intelligent automation empowers organizations to accelerate test execution, reduce downtime and prevent defects. Automation is the only way to effectively keep pace with the testing demands associated with the complexities and rapid-fire release schedules of modern enterprise packaged apps like SAP S/4HANA.
YETI Coolers—a premium manufacturer of drinkware and coolers, and the go-to-choice of outdoor enthusiasts, tailgaters and backyard barbecue kings, experienced explosive growth, creating a need to design and construct highly scalable processes. In response, YETI’s leadership team chose to deploy SAP S/4HANA.
Prior to the S/4HANA implementation, software testing was performed manually on an ad hoc basis, without the support of a dedicated testing team. With the scope and complexity of an S/4HANA rollout, the need for an automated process for testingbecame increasingly important.
YETI sought out a proven automation testing tool that could enable them to prevent business disruption before, during and after their HANA implementation, and support these key business goals:
Build and maintain complex end-to-end tests with speed and ease
Maintain the integrity of existing systems
Reduce production defects
Improve the quality of testing
Validate critical business processes to avoid business disruption
Shorten testing cycles
YETI chose test automation tool Worksoft Certify, the intelligent automation platform of choice for enterprise packaged applications. Using Certify, they implemented an automated process for daily regression testing to ensure flawless SAP business process implementation. YETI also used functional test automation to support customer-facing business processes on its external website.
With Worksoft, YETI was able to automate business process discovery and testing to document and validate business processes and support their SAP S/4HANA adoption. The company’s IT teams used test scripts that mimicked the day-to-day operations of YETI’s core business – allowing testing to be based on as-is business processes. YETI also used automation to conduct end-user and performance testing, eliminating manual effort,saving time and reducing risk.
YETI leaders recognized the inherent business benefits of automation with Worksoft as evidenced by their operations. “The quality of our software systems is how we can say that we’ve been successful with automation,” said James Croaff, software quality and test engineering for YETI.
Automation makes it possible for the team at YETI Coolers to mobilize cross- functional teams to create a robust, unified testing program for SAP and its connected enterprise landscape. After just six months of using Worksoft automation, YETI increased the volume of their test library by more than 500 automated tests. In that same time period, they achieved 85 percent test coverage for their ERP systems and 50 percent coverage of all eCommerce processes.
ERP migrations are highly complex and require significant testing to ensure business process continuity, satisfy audit requirements, and deliver peace of mind for these significant digital transformations. According to International Data Corporation (IDC) analysts, Worksoft automation accelerates business process testing by 76%, empowering organizations to complete testing in mere hours that once took weeks.
Deutsche Telekom, the largest telecommunications provider in Europe, leveraged the value of intelligent automation with Worksoft for their One ERP Transformation program. Dirk Keweloh, senior vice president of Digitization and Transformation for Deutsche Telekom’s Services Europe AG, recently shared the organization’s experience with Worksoft automation at the Gartner CIO and IT Executive Summit in Munich.
Driving Digital Transformation
Deutsche Telekom recently migrated to a single ERP solution to standardize data models, business processes and IT across their complex global enterprise. The ambitious move was designed to give them one truth, one master data and one logic across the services group, which serves more than 175,000 customers with 10 locations and 4 countries.
The organization’s SAP implementation included the migration of complex business processes for purchase to pay, order to case, supply chain, network logistics, and more. With more than 6,000 test cases to be completed in a period of four months or less, company leaders knew they lacked the bandwidth and volume of technical resources to complete the necessary testing with a traditional manual approach.
“We needed some way of reducing the manual effort in rollouts and that’s why I was looking for a new methodology in terms of how I could get rid of these test cases,” said Keweloh.
Automation for Business Processes
During his presentation, Keweloh noted the clear distinction between traditional test automation that’s been available for decades and the flexibility and ease of use offered by the Worksoft platform. Worksoft’s codeless automation streamlines the testing process for business users and their associated processes by enabling flexible scope definition and test execution from the user’s desk.
“What we have here is test automation you can use on the business side. We call it ‘build your own test bot.’ This software from Worksoft is so easy to use. You can train business users so that they’re able to set up their test cases and play with them.”
Keweloh said if the teams suspect there may be specific issues or defects in a certain area of the process, they can easily set up a variety of test cases to run overnight, work to resolve the issue, then run those same established tests again to determine if the resolution was accurate. He said having established tests cases has boosted efficiency and quality as they cycle through rollouts.
“We have ramped up thousands of test bots and can cover 80% of the rollout scope,” Keweloh said. “We are now reducing the effort versus manual test execution and as a result, quality is increasing in our ERP rollouts.”
Documentation Made Easy
Compliance documentation is another added benefit of Worksoft automation, according to Keweloh, who said the platform automatically generates the test documentation necessary to meet audit requirements, which must be satisfied before an ERP rollout can go live.
He added that Worksoft also automates documentation for the small percentage of testing where manual testing is still used. “You can use the tool and just recode what the business user is doing and let that documentation run in behind it. Now the documentation is gone. It can be fully automated by the tool.”
Methodology: Maximizing Flexibility & Reusability
Worksoft automation allows Deutsche Telekom to automate testing across complex, end-to-end business processes. Keweloh said the Worksoft platform is highly customizable and there’s “no programming necessary.” He pointed out aligning the test automation infrastructure with your business methodology is key to maximizing reusability, allowing organizations to further boost efficiency by sharing the automation across teams within the organization.
“Consider how you set up the test scripts. If you want to go for reusability and flexibility, then you need to think about variables in the scripts. You have to train the business user a little bit. It’s not about the tool or technique, it’s a methodology,” said Keweloh.
CoE: Setting the Standard
Deutsche Telekom has reached a level of maturity with their test automation that has allowed them to set up a Center of Excellence for Test Automation.
“We have established test automation as a Center of Excellence from which we drive and automate transformation programs.”
He added that the CoE for Test Automation is “now the basis for providing services and training to others in the organization. Everybody is able to use the methodology and the tools that are in place.”
Beyond ERP: Automating Non-SAP Testing
Deutsche Telekom is now expanding its use of Worksoft automation beyond SAP.
“This is not only about the ERP. Worksoft is very strong on the SAP environment, but you can also use it for non-SAP environments,” Keweloh said. “We are rolling out this technology to other projects as well, including our 35 innovation projects that we run each year. We also want to use it in the web-based area.”
S/4HANA Migration Planning
Deutsche Telekom is currently working with experts from Worksoft and SAP to determine the best approach to use test automation to streamline their S/4HANA migration.
“Having this huge infrastructure in terms of the one ERP, we now need to go for the next step, which is the S/4HANA migration,” Keweloh said. “So, we are in discussions about what is the right methodology for dealing with test automation on the S/4HANA platform.”
Keweloh said he is looking to automation to make Deutsche Telekom’s migration to S/4HANA “cheap, automated, and standardized” by avoiding manual testing and trial and error.
For more information about Worksoft automation, review these resources:
Digital transformation is trending for enterprise HCM for good reason. The 2019 SAPPHIRE NOW® and ASUG annual conference saw a heavy emphasis on HR transformations as companies across the globe are migrating their HCM operations to SaaS-based applications like SAP SuccessFactors® to realize benefits like greater operational efficiencies, streamlined regulatory compliance, and improved employee experiences.
At the SAPPHIRE NOW event, Lufthansa Group’s Programme Lead for HR IT Renewal, Beatrice Kirst shared how the German aviation company is employing intelligent test automation from Worksoft to realize its IT initiative to decentralize testing, democratize automation and modernize the HR experience for its 130,000+ employees.
Moving HR IT to the Cloud
In addition to operating the world’s 11th largest airline, which flies to more than 340 destinations, Lufthansa Group has more than 500 subsidiaries in 140 countries. The aviation giant’s global HR IT landscape encompassed a wide variety of different systems across its vast network of companies. With disparate data sources and applications, Lufthansa Group’s HR operation was unable to align its data, ensure data privacy, or extract comprehensive reporting.
They chose to adopt SAP SuccessFactors EC in the cloud, consolidating data with one source and driving the efficiencies and value that come from digitization and a common platform. To support the transition to SAP SuccessFactors, Lufthansa Group chose Worksoft for their industry leading intelligent automation platform and deep knowledge of SAP.
Why Test a Cloud App?
Even though SuccessFactors is a cloud-based application, Lufthansa Group’s rollout program has several streams related to testing. Kirst said she often asked why testing is necessary since some mistakenly assume that the cloud provider has complete responsibility and coverage for any potential issues. In addition to shared cloud responsibilities between cloud provider and customer, the sheer volume and scope of what an enterprise manages within the application dictate a definitive need for the checks and balances that only testing can bring.
“I’m often asked ‘why do you test a cloud app?’ I mean a lot of companies are using the application. You’re in the cloud so you can be sure that your provider SAP has tested and ensured that functionality is there and up and running, and if there’s a problem we as a customer have given feedback and it’s already corrected, right?” said Kirst. “But it’s more than only having an application running. You’re talking about having a Lufthansa-specific configuration with our own business rules implemented within our HR processes. We have workflows up and running, management services and employee self-services.”
Kirst added that the complexity, sensitivity and nature of the data further reinforce the need to run tests that ensure all is as it should be and mission-critical processes like payroll are executed correctly.
“We will have huge and complex data migrated into the cloud solution. This is not data about how our aircraft performs, how fast it can fly or whatever. It’s data about our main resource. It’s data about our employees.”
Keeping Pace with Change
Testing alone is not sufficient for large-scale SaaS-based HR applications like SuccessFactors. Without test automation, the frequent cycle of change can quickly become daunting for any IT operations group, requiring repetitive, resource-intensive test cycles along with a need for more skilled resources, which are already in short supply.
“SuccessFactors EC gets updated four times a year, two times with more technical updates, two times with functional changes. We don’t want our employees required to do this, every three months, for three weeks, testing the same things again and again,” said Kirst. “Not only would this create errors and frustration among the employees, but we would have to increase our teams doing IT operations.”
Test automation is enabling Lufthansa Group to position their teams to accelerate not only the initial implementation of SuccessFactors, but also to expedite testing for ongoing release cycles, while enhancing business process assurance as they navigate those constant changes.
Dealing with Sensitive Data
With privacy and security laws and restrictions on the personal data found in enterprise HR applications, operations teams can’t simply access the data.
“For testing, we don’t have the possibility to simply copy our production environment to a test environment and start testing,” said Kirst. “That is not allowed. Also, the data access for testing purposes is very, very restrictive.”
Test automation allows Lufthansa IT teams to run their routines to get the results and reporting necessary to know if their business rules and workflows are still up and running and if the data is still correct—without granting access to the sensitive data itself.
Democratization and Decentralization
One of the key benefits of implementing intelligent automation is the ability to leverage what’s already been built. Instead of reinventing the wheel for every test cycle, existing automation can be shared across the organization and teams can add their own nuances or updates as needed. Lufthansa Group is using the global template for their SuccessFactors implementation as the foundation for every implementation and rollout they’re planning, as well as a base for test automation across their subsidiaries.
“All those test cases are implemented within test automation already. This enables us with every rollout to add to the localization, topics, changes and additional functionality like additional business rules or workflows, and then to de-centralize testing,” Kirst said.
Decentralized testing saves time and empowers individual teams to have ownership and autonomy to ensure their individual processes work. Made possible by intelligent automation, this unique, federated approach to testing builds up responsible resources across teams while maintaining enterprise standards and continuity established by democratized automation. That’s why Lufthansa Group is employing intelligent automation not only within their headquarters in Frankfurt, but also throughout all their global business units and companies.
“By decentralizing the testing, the companies themselves can make sure that every process is still up and running after cutover, as they are used to and as they are needed. So they are not depending on what Lufthansa in Frankfurt tells them about how it should work,” said Kirst.
Looking Ahead: Continuous Testing & HANA Migration with Worksoft
With 83 test scenarios defined and 95% percent of the test scenarios for global organization management already proven, the global enterprise is progressing well toward their Q3 2019 cutover. Looking ahead, Lufthansa Group is working with Worksoft to identify additional programs that can benefit from intelligent test automation. They will implement continuous testing to improve test cases and further empower their IT operations to automate tests for SAP updates. Like many global enterprises, Lufthansa Group is considering moving their data to SAP HANA and Kirst said when they do, she sees “good opportunities to use Worksoft test automation to make sure everything is up and working, and also to reduce efforts for testing.”
“We are working on a long-term partnership with Worksoft to establish test automation for all of our HR IT projects and programs to be on the safe side for our future development and digitization projects.”
As you plan your time in Orlando, May 7-9, here are four reasons to visit Worksoft in booth #1255, across from the keynote theater.
#1 Triumphs – Enterprise applications are complex, but our customers have discovered the value of infusing automation into their SAP operations and beyond. Some of these leading global enterprises who are winning with Worksoft will be on hand at our booth to share their real-world success stories. Hear how Worksoft customers Hershey, Microsoft, Glaxo Smith Kline and others are using our intelligent automation to support modern SAP testing, Agile-plus-DevOps initiatives, cloud environments, web testing, and more. View the full agenda of in-booth presentations here.
#2 Trends – Everyone wants to do more with less. Come explore how Worksoft’s scalable platform supports robotic process automation (RPA) to boost speed and efficiency while reducing risk. Our panel discussion with Constellation Research’s Holger Mueller and other experts will explore “The RPA Bonus.” Come hear how Worksoft customers are extending the ROI of their investment with RPA. Our in-booth theater will also feature a session on top-of-mind S/4HANA implementations. Come hear from Worksoft strategic partner InfoSys on how to leverage automation to maintain business process assurance while successfully navigating one of your organization’s most significant transitions.
#3 Transformation – Gone are the days of infrequent updates and take-your-time waterfall updates. Today’s IT environments move fast and release cycles are both constant and inflexible with the potential for massive impact on your critical business processes. Come explore the latest enhancements to Worksoft Certify 11 and see why our industry-leading, code-free continuous test automation platform delivers unmatched agility, risk mitigation and peace of mind to ensure your business processes remain intact throughout the cycles of change.
#4 Trivia – Come play our exclusive Trivia game at Sapphire for a chance to win $50,000! Played online and live in our booth, this interactive trivia game awards points for both speed and accuracy. Bring your “A” game and compete for cash prizes. Every round has a winner and highest score players are rewarded daily. Top 10 high score players qualify for a chance to play our Grand Prize match game in our booth Thursday and must be present to win. Follow @WorksoftInc on Twitter for the game-time notifications.
Enterprises now accept the reality that integrated digital technology signals constantly changing customer experiences, operation models, and business models. In this digital era, organizations cannot afford business disruptions that consume resources and budgets. Yet this is exactly the position many organizations have found themselves in when undertaking large enterprise packaged application implementations like SAP S/4HANA.
Businesses cannot stand still. Updates still need to be made. And processes must continue to run seamlessly, even during massive implementations like S/4HANA. It is imperative that organizations carefully plan and execute their journey to S/4HANA to ensure continuity and mitigate risk. SAP S/4HANA projects not only represent a major technological change but also a change in the way that technology is delivered. Unlike the lengthy waterfall system implementations of the past, S/4HANA is architected to be delivered in smaller, functional pieces using a Scaled Agile Framework (SAFe).
“Digital transformation is about a movement. This is more than a project and requires an ongoing commitment to transforming business models and improving digital channels.”
– R “Ray” Wang | Principal Analyst & Founder | Constellation Research, Inc.
Change Management Transformation
To prepare for the move to Agile methodologies and S/4HANA, organizations are transforming their existing change management processes in order to meet the challenges of S/4HANA and associated applications. And this shift to S/4HANA marks only the beginning of an organization’s digital transformation.
From Project to Value
In today’s climate of constant change, transformation is ongoing, and businesses who intend to thrive in the 21st century will have to continually transform—or be left behind.
This realization presents a shift in thinking:from project to value. Agile assumes cost and time are fixed and views project scope as the variable. But how do you apply this type of thinking to a large ERP implementation where sizeable blocks of functionality may need to be completed in order to promote the feature to the next level of review? What about inter-dependencies of specific modules?
As the paradigm shifts, process automation is no longer considered a luxury. It is a requirement. With each new feature and release, all tests will need to be run. And it’s not just the new S/4HANA processes that will need to be tested. All of the supporting upstream and downstream systems will need to be tested end-to-end. The only way to do this effectively is with automation.
Automate Early and Often
Savvy teams will be proactive and start building test automation long before they have to start building tests for S/4HANA. Why? Because automating known business processes earlier will accelerate ultimate delivery timelines by establishing automation that can be expanded as feature lists grow, rather than waiting until the final business process design and delivery are complete.
Preparing for S/4HANA requires shifts in thinking, both internally and externally. Below are five core areas to consider when planning an S/4HANA transformation:
1. Project Scope
To effectively scope S/4HANA implementation deliverables, obtain a clear understanding of existing business processes, sub-processes, customized systems, affected business units, and users. Determine the number of expected workflows, reports, interfaces, conversions, extensions, and forms (WRICEFs). And clearly define processes, systems and new integrations.
Deconstructing all of these details will require the involvement of multiple users across the organization. To keep everyone on track, create a clear responsible, accountable, consulted and informed (RACI) chart. Now consider the velocity with which changes will be made in Agile and one can see how project documentation and delivery teams can quickly become overwhelmed, which is where problems can occur. Teams may be forced to choose between properly documenting and analyzing a change vs. moving forward with the change in order to meet a deadline.
2. Master Content and Accelerators
Employing the Master Content model and accelerators can be a great way to jump-start deployment efforts while reducing risks. The SAP Model Company has been proven to accelerate project development and discovery times and reduce maintenance and custom development spending. While SAP Model Company is available to help guide your transformation, there is no one-size-fits-all solution for large enterprise SAP S/4HANA conversions. For this reason, many companies engage the help of a Systems Integrator with industry-specific experience that can help them best leverage this approach, while also overcoming challenges unique to their environment.
3. Backlog and Ongoing Change Management
The Backlog and ongoing change management for an S/4HANA project will be much more challenging than dealing with a small development project. The backlog in an S/4HANA project is not constrained to activities related to development, but also consists of things like data support, training, deployment, Change Control Boards (CCB) Reviews, compliance audits, and more. To complicate things further, the backlog may be composed of requirements from multiple teams with conflicting priorities. How to schedule, prioritize and manage communications across the multiple teams requires more orchestration and involvement at the executive level than what is used in most Agile practices today.
4. Centralized Management and Approvals
Some level of centralized management is necessary to any large-scale enterprise project. Executive support is critical to ongoing project success and periodic reviews need to be scheduled. Engaging this level of support frequently requires additional documentation, which can bein direct contrast with the core principles of Agile.
In addition, the nature of centralized management itself must be adapted to support the Agile delivery of services. Change Control Boards (CCB) must be available to review new functionality when it is ready versus reviewing based on a set schedule. Security and compliance reviews also need to be added to the backlog, while testing teams will need to have tests built and ready to run.
5. Exhaustive Testing
When budgets run low and deployment windows get tight, testing is often sacrificed. Failing to thoroughly test a system or understand the extent to which a system may fail is a recipe for disaster. At a minimum, organizations could face expensive rework. But more devastating results are possible that could negatively affects the organization for years to come.
Management of integrated testing can also be challenging since much of ERP testing requires the involvement of multiple teams. Deciding who will own and maintain automation in the long-term becomes a question that must be addressed outside of Agile practices.
Bottom line? Understanding, ongoing change management and expansive automation adoption are the keys to successfully adopting both Agile and S/4HANA. And the time to start thinking about the best practices, tools and teams needed to support the ongoing change management needed for S/4HANA is now.
Evaluating your S/4HANA Readiness? The following resources provide additional insight into S/4HANA readiness and how automation helps organizations meet the challenges associated with S/4HANA projects:
Avoid Overly Simple Tools When Testing Complex Systems
With so many different tools and methods in the market, how do you choose the right test data management solution (TDM)? By now, you’ve probably realized there is no simple answer to this question, especially when looking for a solution to test complex end-to-end business processes and systems like SAP that involve a mix of both web-based and traditional Windows-based UIs.
SAP enterprise landscapes are highly complex and interrelated. To get good test data, referential and logical integrity must be preserved, and not all relationships are explicit. Data has to be correlated across tests and systems. The data needs to have the context of the business process being executed, positive or negative test, standard orders, or out-of-stock orders, etc. Most test data management tools on the market generate data based on a single set of functional requirements and lack the ability to correlate data across a horizontal end-to-end business process test.
This is probably why we see most clients continue to struggle with test data. They continue to invest time and money trying to leverage tools and approaches designed for single-app, vertically focused test cases and then wonder why they continue to spend so much time and effort generating test data.
What Should You Look for in a TDM Solution? Effective enterprise test data management solutions enable users to see the data relationships across test cases and end-to-end business systems. These solutions allow visualization of the data that is available, the data that is missing, and the data that needs to be created. They need not only to generate data for functional tests but also for horizontal end-to-end business-process tests. This means being able to support tests that cross a plethora of enterprise applications with multiple UIs, including mainframe, ERP, CRM, web, mobile, personal apps, and others.
TDM may not be simple, but evaluating a TDM solution’s ability to support end-to-end business process testing really comes down to two things:
1. Ability to Understand a Supporting Systems’ Data Perspective The test data views need to be generated from the perspective of the system under test. Data must not only be in the right format but also be validated with the system under test at the time of the test. To ship an item, it must exist in the warehouse; it’s not enough for the data to be just in the right format. The test data dependency on the data in the supporting system must also be accounted for. Most testing tools take a very limited view into test case data and fail to do this. A Gherkin script describes the data being entered into an API or web page, but it does not describe the back-end systems’ perspective of that data.
2. Ability to Correlate Data Across the Business Process Data must be correlated across the entire horizontal end-to-end business process. The business process needs to have the context of the job to be done, positive or negative test, standard order, or out-of-stock orders, etc. A successful order means the product in an order does exist in the warehouse, is in stock and has the right price when the test runs. In contrast, for an out-of-stock order the product has to exist, but the quantity on hand has to be a specific value.
Without this understanding of the context, the data is just optimized random values. A combinatorial math calculation like some TDMs use does not understand this context, making it too simplistic with the test case data representing only a single vertical transaction. Data without context is less valuable and, in some cases, simply useless.
For more on the various approaches to test data management and what to look for when selecting a TDM solution for testing enterprise applications like SAP, read the white paper “Enterprise Test Data Management Practicum vs. Marketing Hype”.
Enterprise organizations across the globe are looking for new ways to boost business efficiencies and enable new innovations faster. UPM has discovered the benefits of incorporating continuous test automation to achieve their goals. They recently shared their digital transformation story at TestExpo 2018 in Helsinki, Finland
A Global Industry Leader
UPM is a global leader in the forest-based bioindustry. The $10 billion EUR business focuses on the efficient use of sustainably sourced, renewable forest biomass. From furniture to food packaging, their wood and paper materials are widely used in home and office environments throughout the world. UPM has 54 production plants in 12 countries, serving customers in 120 countries with products that are engineered for safe and sustainable everyday use.
Drivers for Test Automation
A progressive industry leader like UPM is constantly evolving as they adapt to new research and ideas, keeping their products and services at the forefront of innovation. System updates and process changes are key to these transformations. The greater the impact of change, the greater the testing effort required to maintain smooth business processes throughout their operation.
For most enterprise organizations, the resource burden of large-scale change necessitates increased speed and efficiency of testing. UPM leverages SAP for their operations and engages in new SAP releases for S/4 system upgrades annually, as well as quarterly SP levels that also require extensive regression testing cycles. In addition, mergers, carve outs and/or complex changes to their operating model could occur as often as once a year, on average. It was this landscape of change and complexity that drove the need for automation.
Shift to In-House Automation
Given the level of effort involved in each testing cycle, UPM recognized the value of shifting to automation. They determined that they had an opportunity to automate 70% of their testing and chose to transform their operation with these goals in mind:
Speed testing processes and significantly shorten test cycles
Gain business agility through progressive, immediate implementation of high-impact changes
Free resources previously assigned to manual testing to focus on development and operations
Improve test effectiveness and quality
The UPM team evaluated two paths to automation. One would require a multi-solution approach to outsource testing. This option is generally not only the higher cost option, but also limits control of the overall testing process and increases opportunities for error.
The other option was to implement test automation in house, which offered maximum control over the process as well as maximum total cost of ownership (TCO) benefits. It was the TCO of a clear business case that lead for an in-house test automation.
Methodical Approach Yields Results
Implementing an in-house solution for test automation can reap significant rewards, but those rewards can be fleeting if the organization doesn’t make a concerted effort to implement the solution strategically. When UPM chose their test automation tool they were backed by a dedicated and motivated team that executed the transition over a seven-month period with a methodical approach and ongoing support from the selected test automation vendor. This approach resulted in 808 scripts to-date, reaching a continuous automation level of 78%. UPM’s shift to automation also includes the use of a test orchestration tool that delivers centralized monitoring, test acceleration and maximum control over test order, dependencies and scheduling.
Positioned for the Future
With continuous test automation enabled at their enterprise, UPM has elevated their IT operation and paved the way for faster adoption and delivery of new innovations. They now have the ability to drive efficiencies for library maintenance, training, infrastructure maintenance and test management. They can also execute test automation at will, which can support expedited system recovery in the event of unplanned service breaks.
Looking ahead, these efficiencies mean UPM is well equipped to support agile development, as well as a future transition to SAP S/4HANA.
For more information on the benefits of test automation:
Many companies have built simplistic, vertical TDM solutions that fall short when it comes to meeting the complex data needs of testing modern-day business processes that traverse multiple applications and have numerous data dependencies (e.g., horizontal end-to-end business processes). In the worst case, what is described as “test data management” is actually just a set of open-source buzzwords describing behavior-driven development (BDD).
Users can provide a list of data values to select from as part of scripts. This “list” becomes the test data repository. Using the Gherkin language, the user writes the test case, including data variables, using a simple English-language script. But within the simplicity of the script, the complications arise. The lack of detail surrounding the script leaves the meaning up for interpretation from user to user. Even though the testers understand which list they should pick the data from, they may or may not understand what each of the values of the data within the list is and what the correlations are with the other steps in the test.
Consider the following example, in which a business analyst supplies the testing team with a simple script with pick-list data for testing the order process for a bicycle.
When you select a value from the pick list in a Cucumber test, you have to ensure you are picking a valid value. Does the product actually exist in the warehouse that you selected? Is there enough stock to fill the order described in the test? Will the selected data work horizontally across the business process?
Does creating and maintaining these types of pick lists really provide anyone with any value? At the end of the day, this is still manual testing and manual data generation, and it is labor intensive, tedious, and prone to human error.
We have to remember some of the guiding principles in modern IT before we just assume the data exists that the users expect. We can’t just copy production data into test anymore. Personally Identifiable Information (PII) is much broader than you think. Any customer data could fall into PII problems, which means a business person helping describe the data can’t think of the data they have in production because it is different.
Complex end-to-end process tests require the data to be synchronized across the process. Consider a simple order for example. How many do the various devices and applications (mobile, web, pricing, and ERP) have to sync to the supporting catalog? Typically, each system has their own system of record and just because you want to order the part does not mean the pricing is synced between the web shopping basket and the ERP.
TDM may not be simple, but evaluating a TDM solution’s ability to support end-to-end business-process testing really comes down to two things:
Ability to Understand Supporting Systems’ Data Perspective
Ability to Correlate Data Across the Business Process
Why Automated Testing Is Key to Business Continuity During the Busiest Time of the Year
As we enter the holiday season with full force, retailers and manufacturers around the globe are hoping they are prepared to meet the demand. From clothing to cars to chocolates, it’s the most wonderful time of year for many organizations’ bottom line.
Whether these holiday purchases are happening on the store floor or through online channels, all the transactions are going through at least one electronic system. And then there’s manufacturing, supply chain management and every other step in between.
Many companies that depend on a successful holiday season to stay in the black have experienced system upgrades and changes throughout 2018, no doubt in preparation for the frenzy these last few weeks of the year are expected to bring. But will their systems support the surge in volume? Have the updates and changes been fully tested throughout their delivery cycle? Can their mobile apps withstand Cyber Monday mania?
The new pace of business change demands constant product deployments and technology updates, and this demand only increases during the holidays. As companies seek to meet consumer demand for speed, agility and convenience, more are adopting an Agile+DevOps approach to expedite innovation and speed deployment. With these updates comes changes, both large and small, that can impact the rest of their systems and related processes.
For large enterprises, the resulting speed and volume of change can be significant. And so can the potentially negative impact if something in their delivery chain gets broken. No one wants to experience their e-commerce site crash, their mobile apps malfunction or see their retail store flooded with a flurry of angry shoppers whose orders cannot be processed.
These holiday headaches can be avoided with proactive testing. Continuous test automation can help ensure that your business processes remain intact by empowering you to schedule and repeat comprehensive testing across your systems, identify potential issues and resolve those issues before they impact the user.
Without automated testing, companies are spending too much manual time building process assurance and are taking on too much risk. In this climate of rapid innovation, both can be catastrophic to an organization. Among the consequences are downtime and disruption to valued customers.
Building an automation strategy to ensure business process excellence during the holidays is the best way to mitigate risk and potential failures. Automated testing not only frees up analysts’ time, speeds delivery, and allows flexible test scheduling, it also provides a higher level of quality assurance with more comprehensive testing and reporting capabilities—so you and your IT team can experience a peaceful holiday season.
Happy Shopping from the Worksoft Team!
For more information on the benefits of test automation:
Here is a look at the top five questions I consistently get when talking to clients about leveraging Worksoft to support Agile-Plus-DevOps environments:
Does Worksoft support Agile-Plus-DevOps environments?
Can Worksoft Certify tests be run from Jenkins and other continuous integration tools? (parts A and B)
Can build numbers be tracked in Certify results?
How does Worksoft support continuous testing?
Which ALM and defect tracking systems does Worksoft integrate with?
We’ve also included a number of additional resources we’ve recently created to help clients on their journeys.
#1. Does Worksoft Support Agile-Plus-DevOps Environments?
Big YES on this one! We have a number of customers who have adopted Agile and DevOps and are using Worksoft every day to create and execute tests across custom, mobile and packaged applications. SAP’s internal IT team wanted to apply Agile testing techniques with a “test early and often” approach and is using Worksoft to help do this. At a high level, customers leverage stand-alone capture to enable business users to create documentation and automation, and they use Worksoft Certify to harden the automation. Then, they execute tests via their CI tool chain or by using Worksoft Certify Execution Manager to execute continuous testing cycles.
We recently hosted a webcast delivered in conjunction with Forrester analyst Diego Lo Giudice in which we go over the “why” Agile and DevOps for packaged apps in the first part of the presentation, and then we cover the “how” with Worksoft in the second half.
#2. Can Worksoft Certify Tests Be Run from Jenkins and Other Continuous Integration Tools?
CI/CD Demo: How to Run Worksoft Certify Processes from Jenkins at Scale and in Parallel - YouTube
Part A: Yes! Worksoft offers command-line support from Certify for executing a test and an out-of-the-box integration for a RESTFUL API that can be called from Jenkins, Microsoft Visual Studio TFS, Micro Focus ALM, SAP Solution Manager and others. Users can run Certify tests from CI servers (Jenkins, TFS, etc.) and pass build numbers for reporting via command line whether the CI server and test lab are the same. This approach works in environments in which security is not an issue and the agent machines used for executing Jenkins can be left logged in with a console up and running all the time.
The Worksoft best practice is to use Certify Execution Manager to execute tests, which solves a number of the environmental problems of location, user screens and varying devices.
In most large enterprises, the build servers are not in the same place as the “test labs.” The option of spinning up virtual machines typically doesn’t work for large packaged app environments. Usually, dedicated hardware, databases and applications are staged in pre-prod to run tests on. Certify Execution Manager allows Jenkins to make requests over the network to run automated, on-demand tests in parallel across different remote test labs. Jenkins makes the test request, and Execution Manager distributes and runs the tests across the resources in various labs.
Part B: Why use Execution Manager with or instead of Jenkins? Why should you not just run Certify tests from the command line in Jenkins?
Yes, it is true that Certify is super-flexible and you can run tests from the command line, but there are challenges to doing this in large enterprises:
Access – When you run a UI test on a Windows device, there must be a user logged in to the machine and the screen can’t be locked so you get all your screenshots in your test and business process procedure reports. Execution Manager handles access and auto-logging in to machines.
Security – Try asking your security team to give you a Windows desktop that is always logged in, has the screen saver turned off, and will never lock because of inactivity. Execution Manager not only handles logging in to the machine but also logging out of the machine after the test has been run.
Orchestration – Most customers want to run their tests on a wide variety of devices that mirror their production environments. This includes a wide mix of OS versions, application versions, and user security and roles. Execution Manager can request that tests be run on different machines so you can verify that the differences in OS or application patch versions don’t break the end-to-end process.
Parallel Test Execution – Unattended parallel test execution is enabled when the CI server and test lab are the same, or via REST calls to Execution Manager to schedule and run tests in remote or cloud-based test labs.
ALM Updates – Bidirectional integrations with ALM systems can be used to execute Certify test processes, view the results and leverage all the ALM functionality for test management. Certify test results can be viewed from within an ALM, showing both the test execution summary as well as details for each test step.
#3. Can Build Numbers Be Tracked in Certify Results?
Yes, and we do this internally when we test our own products. We like to frame this as “drinking our own Champagne.” When a Certify process is run, a user can specify a result attribute, which can be set when running from the UI, command line or Execution Manager. When we run our regression, our internal tests for Worksoft Analyze, Execution Manager runs them in multiple test labs and with multiple browsers. Our result attributes have the Lab (Dev, QA1, QA2, production) and the build number assigned by Jenkins. Our reports then show the pass/fail information by environment and build number.
Demo: Process Compare and Merge in Worksoft Certify
Worksoft makes it easy to harden and variabilize tests so they don’t break. A Certify test never needs to be rerecorded when the application or process changes. You can easily open an existing Certify process to add, modify or delete steps when required. If an application object changes, an update can be performed once, and all related tests will update automatically. Improve test reuse even more with process compare. It’s easier than ever to consolidate and reduce test maintenance by quickly evaluating two processes to determine whether an existing test provides identical coverage.
Typically, testing done within an Agile-focused sprint focuses on a specific story and small piece of functionality. Testing at the feature level is more extensive and looks at the larger workflow. The challenge, then, is syncing these two types of tests. New features tests run as part of CI cycles also need to be passed back to run as part of CI cycles so that a larger net can be cast to ensure stories don’t conflict with one another or cause problems within the larger epic. Worksoft enables both CI and CT and optimizes ROI gained from test automation efforts.
Demo: Daily Continuous Testing with Worksoft and Jenkins
Running Tests Daily – In addition, our services teams have come up with a set of best practices for enabling continuous testing with Worksoft, including:
Setting Up Test Data – Certify processes can be used to set up data and then to run the tests against the data. For example, to do a delivery and create an invoice, don’t think that there is data magically in the system or that it will magically show up. Users can have the tests create their orders so they can then deliver and invoice.
Verifying Workflows – Certify processes can be used to verify real user workflows and end-to-end workflows versus simple atomic units of work. There is huge value in running tests the same way as the end users who use the systems to run their businesses.
The third best practice is to schedule tests via Execution Manager to run daily or weekly, based on how many changes happened that can’t be correlated to a change list item. If you have only 10 changes a week, run them on Fridays; if you have 25 changes a day, run them daily.
#5. Which ALM and Defect Tracking Systems Does Worksoft Integrate With?
Diving into the conversation a bit more, the follow-up question I get is, “How do I keep track of my requirement to test when all people talk about is epics and stories,” which is then followed by, “I think we’re going to use Jira for this.” A common misconception is that Jira can be used to replace an existing ALM. Why not? Jira is cloud-based and less expensive — easy decision, right? Unfortunately, it is not that easy. The workflows for managing requirement and code flows are very different from those for managing test plans.
The Jira green plus icon says add epic, story, task or bug. There are no test plans, test cases or test results in Jira. To get this functionality, a plugin must be used. In addition, multiple other plugins will need to be added to match the functionality of a Microfocus (HP) ALM. In the end, you have a plethora of different tools that have to be integrated and managed, which may end up costing more and increasing complexity.
The good news is the classic ALM platforms now have Agile Scrum workflows along with just Waterfall (i.e., HP ALM Suite and IBM). Teams can continue to leverage their existing ALM systems as the system of record, and for audits as they adopt Agile. A more modern Agile planning tool, such as CA Agile Central (formerly Rally), provides not only organization of epics, features, stories and tasks, but it also can map the supporting test plans, tests, cases and sets. This is great for project managers and enables teams to keep all the project resources together.
Looking to learn more on how to use Worksoft? Check out Worksoft University, which features a number of free, self-paced online classes.