This is a project by Rob Lambert and Joel Montvelisky where we are answered a number of interesting questions that we get from testers constantly and that focus around a number of different areas of our very interesting profession.
Unless you have been stuck under a rock for the last 15 years, and even if you have been stuck under a rock testing, then we are sure you have been doing some sort of Exploratory Testing as part of your work.
We all do ET, but we are getting ahead of ourselves…
There is an approach to testing called ET and it is important to know how to use it, when to use it, how to manage it, what are its strengths and some of its weaknesses and we want to talk about this today…
Let’s try to go over them:
What is exploratory testing?
James Back, one of the founding fathers of ET (alongside Cam Karner), defined exploratory testing as understanding, reviewing and testing software at the same time.
Everyone, every time does ET in the most strict sense of the word…
We do exploratory testing all the time.
ET is working with software in order to test it, and continue to understand it as you go along
Even when you have a script, every time you test a button you will look at its surroundings and check into that; so you will be doing exploratory testing
You will be learning the system and understanding it, at the same time you are testing it
Testing is a spectrum, on one side of which we have scripted testing and on the other side exploratory. When a human is involved in the process, it will always be somewhere in the middle of that spectrum. In most cases it will lean towards one of the sides, but it will never be explicitly on one side
How do you go about managing Exploratory Testing
One of the most common ways is session based test management. This will serve as a framework to direct you on what to look at.
Limited session time, with clear goals and aim
In the session, you raise issues, ask questions, write notes. The questions raised on the current session can then create opportunities for following sessions
The test charter points that guide your session can constantly be updated and that will keep your sessions relevant
Evidence is crucial: it will be used later for reference
There are a selection of tools to choose from for recording session based tests
When does it work wonders?
When requirements are vague
When testing complicated or complex products
When you’re curious about something….
When you’re testing
When does it not work at all!!!
It might not be possible to replace scripted testing in some environments
But I’ve rarely seen it never work
The tester’s experience and pre-session guidance are big factors
how to work with distributed-teams/outsourcing/offshoring/crowdsourcing
Lately I was on a trip that took me to Tokyo, SF and NJ. Took part in a couple of conferences and got to visit many customers and potential customers of PT.
I was traveling a lot in the last month or so, and something that struck me was the fact that regardless if I was sitting with a team in Tokyo, San Francisco, or New Jersey; one thing in common that I saw in most of the organizations I visited, was that most of them were working as a distributed team.
The guys in Tokyo were working with external testers in Vietnam or Malaysia, one team was actually an international team composed of people from 5 different countries (and 4 mother-tongues). And in a similar way, the teams in the US were working with either Eastern Europe, India or Asia.
This is not something new, I see teams all over the world with either outsourcing, crowdsourcing, offshoring, etc.
The world is now a Global Village, and this is bringing many good things, but also quite a lot of challenges.
Let’s try to go over them:
1. Time differences and Geographical separation
This is a trivial one. Since we are sitting in different places and many times working at different times and even days, we need to find a way to coordinate the work and organize the tasks done by the entire team.
2. Language barriers
So, it seems that our assumption that everyone speaks English around the world is not correct. For example, when traveling to Japan, I realized that even though many people do understand English, most people do not feel comfortable talking in English. Even those who could understand it, preferred to have a translator present to make sure they understood everything. And we are not talking about a 3rd world country, this is JAPAN!
3. Culture differences
I remember working with an offshore team, who did not feel comfortable telling me the product was in a really crappy state, it was not in their cultural norms to complaint with people from higher ranks, and so my surprise was to realize 2 weeks before the release that we could not even install the product in our staging environment without the help of developers. Because of this, the project was delayed for about 3 months!!!
4. Complex relationships with external companies/service providers
I hate politics, and whenever you work with an external company or service provider there will be no way around it, you will need to work with them. When a person works for another company, regardless of the person and the nature of the company, there will be frictions and in some cases even conflicts of interest.
5. The feeling of not being part of the main process or team
Many years ago I had my main team in Tel Aviv, but I had a second team that was just as big working in Odessa Ukraine, and these guys had the disadvantage of working away from the action of the development team – this caused a lack of motivation and a lot of people just leaving the team earlier than planned…
Bugs are an important part of the work of testers, but for some reason we tend to take them for granted, and don’t invest the proper value and thoughts to them.
I am not saying they are the most important part of the process, but they are an incredibly useful output and measurement if you manage them correctly and know what you are looking for.
Bugs are also the cause of anger and miscommunication between teams, and when approached lightly can cause as much harm as good.
But let’s be clear, it’s fairly inevitable that your software has bugs in it…..almost all software does.
Today we want to continue talking about some important points that relate to bugs.
Do all teams need triage?
What is triage and how come we are using a medical term defined in war time to handle bugs in development practices???
Triage is an bureaucratic overhead, I try to avoid it as I try to avoid unneeded meetings.
I prefer to have a mechanism to escalate bugs we do not agree on their priority. BTW, triage should not determine the severity of a bug, only the priority…
Severity, Priority or Both?
The answer is depending on what it is needed.
Just remember that priority is subjective, while severity is objective.
If you can, create a severity table… Use even number of category, not uneven
What is it with this “zero bug backlog” idea or trend?
Nice idea, a bit overhipped, do not think it is an objective but maybe an indicator of a very agile process. Still zero zero zero backlog sounds a bit strange as we may want to keep track of things we do not want to fix right now, specially when working on iterations… So ok, we can make a story for it, but then we are just talking semantics.
All bugs should be fixed
Thanks for listening.
As usual let us know what you think in the comments or on Twitter
Don’t forget to like and subscribe to get more content like this
Culture is nothing more than group habit – it’s what people do every day. So in order to change a culture to one of a “quality” one, you must identify the behaviors that lead to good outcomes and encourage these throughout the organization
I first heard about the 1% approach when I read about the amazing success the British Cycling team had at the Beijing Olympics in 2008. Before this time we’d only won a single Gold medal for about 70 years. Then, after some 1% work from the new head in charge, Sir Dave Brailsford, Britain won 10 gold medal in Beijing – and another 10 4 years later in London. And then he had the same impact on the cycling in the Tour de France.
So what did Sir Dave Brailsford do?
He, and his team studied three pillars of cycling; strategy; human performance; and continuous improvement. And they identified lots and lots of areas for tiny improvements. Each of these very tiny improvements, from rubbing alcohol on tires for better grip to testing which pillow gave the cyclists the best night’s sleep. All clocked up. Marginal Gains.
We often assume change should be one big whopping hit of change, but the reality is these tiny constant improvements over many areas all compound and adds together to create huge change.
Let’s look at this from a testing perspective, you could apply changes to any aspect of your world, from environment suitability, tester skill, communication between teams, clarity of requirements, tooling used etc. Tiny changes, over time – add up
When things go wrong understand why they went wrong. Get to the root cause and understand the systemic failures that happened. It’s rarely a single thing that causes a failure or mistake to happen.
Asking why 5 times can often lead you to the root cause, but don’t ignore the failures at each stage of the ‘why’, that lead to the final impact. Each time you ask why and get an answer as to what went wrong, there is a chance to make your world better.
Sometimes, it’s a person who made the mistake. It’s not a witch hunt. It’s not a blame culture, at least it shouldn’t be. As before though, it’s rarely a mistake by a person that causes fall out – there are often system failures, processes and communication challenges that helped the failure happen. It’s hard to run these. Go and search for how to do them, and then run them for everything that goes wrong. And then fix the problems that lead to them. Also, don’t forget, in most organizations people have no idea why things went well either. So run them for things that go well that were unexpected.
Testing is an activity, not a phase. On that note, Quality is a mindset and a priority, not an attribute to be measured
Testing can be done by anyone
Quality is more than lack of bugs. Testers can (need to?) influence testing at all levels – Don’t wait to get that job title, push the bar yourself. Yep – test the system and process you work in too.
The flip side of the coin – It is important to understand that each company needs a different level of stability and quality before they can release the product.
Joel Montvelisky, Chief solution architect at PractiTest – test management tool, and Rob Lambert, Director of cultivated management, a tech consulting company with today’s topic: Is software testing more about testing or programming?
As the testing world evolves, the lines between tester and developers are starting to fade.
In the past, automation tests were used as an easier way to perform regression testing. Test automation evolved since then and became a big part of the testing process.
In today’s testing world, in order to write good test automation scripts, you need to have programming abilities.
You can use test automation in almost every aspect of the testing process: functional testing, load testing, stress testing, generating testing data, enabling testing environments, etc.
Automation/scripting is a tool that should be used to save you time. It will not do the testing for you but will help in the checking process.
The testing process will never solely rely on automated tests. Some defects can only be found by a human interacting with a system.
Today, especially when working with SaaS environments, the best use of a tester’s time is working on the instrumentation – creating alerts, parsing logs.
In the test automation process, the person that should dictate what work should be done is the tester, while the person that is doing the actual work should have programming abilities.
Collaboration between testers and developers can be the key for a great test automation process.
Developers that write automated tests can get familiar with the testing process, which will allow them to participate further in the process.
Writing a good script should be treated as a development project. You need to invest time in technology, tools, and planning, it should be modular and it should run with a good framework.
If you think you can put in the effort, invest your time and learn how to code you should, but coding is not for everyone and you can have great value for your team regardless.
Joel and Rob discuss what skills and roles testers should focus on for career development.
There are so many different career routes for testers nowadays. In order to be able to choose what niche to focus on it’s recommended people try a plethora of different aspects of software testing and find one that resonates, or they’re interested in, or one that leads them down the path they want int he longer term.
Why do we need Niches in testing?
Different risks and different projects will require different types of testing to ensure a specific aspect of the product is being written/developed/planned/deployed correctly.
Niches or specializations, allow testers to expand professionally. Many times a tester asks others or themselves what is there to learn in testing? And one answer is Niches that he or she can become more versed
Niches are more needs than decisions. Many times we need to learn something or actively study it on the fly, because of an active need of a project, and that is how we become experts on this or that niche.
What niches of software testing are open to testers?
“Internationalization is the design and development of a product, application or document content that enables easy localization for target audiences that vary in culture, region, or language.
Localization refers to the adaptation of a product, application or document content to meet the language, cultural and other requirements of a specific target market (a locale).” – Read more:W3C
Niche 3 – Accessibility
Checking website accessibility is one thing, ensuring its complaint is another.
Some tools are good indicators and can pick up obvious binary rules
Human judgmentt is best
Read more – https://www.w3.org/WAI/standards-guidelines/wcag/
Niche 4 – Finance
Niche 5 – Medical or Healthcare related
Niche 6 – Hardware / Kit Testing
Niche 7 – Mobile & Devices
Niche 8 – Instrumentation & Data analysis
Niche 9 – Performance
In recent years performance testing has become low touch, component-based and ultimately – done in production where possible with rollbacks, clever monitoring, active alerting and small incremental product changes – which makes it easier to diagnose performance problems
Niche 10 – Automation
The biggest area that testers seem to focus on. Automated checks belong to the developer who wrote the code. There may be a small team of people enabling the tests to run – such as providing environments, pipelines, training, and reporting. And one could argue that they are in that case automation testers. However, developers are much better (and often willing) to build these systems. After all – test code should be treated as production code – who better to write the code than the programmer?
On the other hand – who better to design tests and think about the questions testing is trying to tease out – than testers. So if both can work together that’s best.
Niche 11 – Infrastructure, environments, and deployments
Niche 12 – Certifications and Compliance and audits
Niche 13 – Testing Training and Consulting
Requires a different set of skills than testing skills. It’s mostly about relationships, sales, communication and information sharing.
It is important to remember that Niches are not carved in stone, they appear based on needs, evolve or morph based on how the technology and methodologies change and disappear once they are not needed.
Testers should not be shaped, we should strive to be ‘Broken Combs’ with different levels of knowledge in different areas or niches.