AskTester is not just another forum. AskTester is a community of professional testers where we can freely ask questions about testing, speak up your opinions, show your interests and sharing stuffs. Not like forum, AskTester will make sure your voice is listened.
For my exit criteria i want to be able to include the percentage of passes and failures that are acceptable to sign off testing e.g 80% passes therefore testing can be signed off but how do work out the percentage that is acceptable?
I don’t know you but if you like me, you probably heard (or overheard) about respect and how to earn it.
It seems respect is critical for people.
Why is that? Well, because… it is.
Actually, earning respect is critical in every aspects of life. When having respect, you have everything. Having no respect? You have nothing. Well, it’s not that extreme but you have to agree with me having respect is one of the most valuable assets.
But respect is strange. It’s hard to earn it but easy to lose. You can do everything right but one good day you do one thing wrong and boom…your respect is gone.
That one thing wrong is different from person to person but to me it’s addressing me wrong or writing my name wrong.
Call me picky if you want but if you don’t write my name correct, you are far away from earning my respect.
My name is “Thanh”, more exactly is “Thành” written in Vietnamese with a grave mark. If you address me as “Thanh”, it’s no problem. I’m perfectly fine. However, most of foreigners often write my name as “Than”, “Then” or “Thank”. It’s a little bit annoying but understandable to me because I realize my name is pronounced close to “Than” or “Thank” to most of foreigners.
Again, it’s perfectly fine to me. When it comes to Vietnamese name, it’s common.
When people address me wrong, I often send a gentle reminding so they can address me right next time.
Now if that same guy keeps addressing my name wrong again and again, that’s another story. That’s unacceptable to me.
I felt like this…
“Take it easy man, people make mistake all the time” or “it’s just the name, man” you may say.
You may be right but to me addressing people’s name right is important. This is especially important in context where people do not work face-to-face, they just knew each others, or communication is made via email or online chatting.
Needless to say, the name is always considered one of the most valuable things a person can possess. The name often binds to the identity, the brand and in some cases…a story. It doesn’t mean (much) to you but it means ( a lot) to them.
So if you write my name wrong again, again and again, it gives me perception that you don’t care to do it right and because you don’t care to do things important to me right, how you can earn respect from me?
Here’s another reason why I stress much on addressing people’s name correctly:
In case you don’t know, most of Vietnamese names has the meaning with it. If you don’t write it right, it changes the meaning. If you don’t use the right tone, it changes the meaning and in some cases to something…well…bad.
For example, my name is “Thanh”. If you mistakenly write my name as “Than” or “Thank”, it means differently. Fortunately, in my case it doesn’t change the meaning to something bad. However, in other cases, it’s not so pleasant.
I used to experience a case where a lady has name as “Nga” (meaning swan in Vietnamese) but mistakenly written as “Ngu” (meaning stupid in Vietnamese)….Uh oh.
This may sound strange to you but if you’re reading this and you’re Vietnamese, you would understand what I’m saying here. Vietnamese is very complicated in terms of writing and pronunciation.
If you’re foreigners, it’s understandable that it’s very difficult to spell or pronounce Vietnamese name correctly. However, if you’re doing business or communicating with Vietnamese, please keep this in mind when you address a Vietnamese’s name. If you write it wrong, you may change people’s name to something different.
Another mistake that really bothers me is that I see that some people has habit to write people’s name all in lower case.
It’s ok to write my name as “Thanh”, or “T” but it’s NOT ok if you write my name as “thanh” even in informal context.
Oh man, who’s on earth to write people’s name in all lower case?
More often, I receive excuse like “it’s faster to write like that”.
Really? Excuse me I’m not aware that you are that efficiency in work. How much time you save when writing people name in all lower cases instead of capitalizing the first letter.
What is my suggestion?
In case you are not sure how a name is spelled or pronounced, just ask. Most of the time, people are willing to tell you how to write or say their names correctly. The important thing is to remember that and avoid to make the same mistakes again. In any case, when you’re not sure, just ask.
For those foreigners, my suggestion is that if you have people’s permission to address them by an alias which is easier for you to address (such as Mike, Bob, etc), please do that. However, it’s best to address their real names in their languages.
For those who have the habit of writing people’s name in all lower cases, please don’t do that. It doesn’t save you time by doing that, worse, it doesn’t show your respect to people.
Now if you see someone addressing you wrong, please gently remind them. It’s probably because they are not aware of that. Don’t take that for granted because writing people’s name right is the first step for a respectful and long-term relationship.
So please, address me right because I address you right.
I got Associate in Computer Science, and finally figure out that this major does not fit to me at all.
First, I am not passionate or any ambitious to it.
Second, It is really a hard major to chew and get along with. I studied very hard to get enough C to pass all CS courses. My professor said if want to be a developer, we do have to be good at math and critical thinking. I think I am not in this category.
But I really don’t want to waste my money and effort that I put all in CS. I am really stressful to thinking about my future then. So what should I do with it? Should I change my major or study another major which is completely different from CS? Now my age is getting closed to 30. I am considering to Information System (more about in Business than in technie).
I really need to hear to your recommendation. Hope I receive yours in soon.
As a Software Quality Engineer, I have been through many interviews in my early career and the most frequently asked question was about software testing life cycle. I tried to cram the answer but soon I realized that it is a very wider concept and I have to consider the complete picture before deciding what it actually is.
So I decided to help my readers to understand the concept in a simpler way.
So what is Software Testing Life Cycle (STLC)?
Software Testing Life Cycle is a group of circularly arranged testing activities, in a specific sequence to understand and test the software in a structured way. These activities are mostly coupled with one or more software development stages and are not bound to testing stage itself only.
Why do we need it?
A lot of people might be wondering that testing is a very simple process in which you can just take a product and test it whenever you need, then why does it have a rocket science behind it?
Well, it’s not as simple as it seems. Always remember that a good process always leads to a good product. We require following a systematic approach towards testing because it is a serious job and no risks should be taken in it.
In order to help you understand better about STLC, let’s have a look at some generic stages of STLC in most widely used software development process models.
Note: Please also noted that these stages are not fixed ones. We can always make a hybrid STLC according to the nature and needs of the organization and the product.
Software Testing Life Cycle in Water-fall Model
In a traditional cycle, we can have the following concrete stages and there is only one testing cycle in this approach and no two stages can be executed parallel.
Requirement analysis: The team first double checks the clarity and of the requirement and discusses the agreed requirements. Then they thoroughly study them to get a grip on the scope of the system and mirror it in a Requirement Traceability Matrix.
Test Planning: The techniques and approaches of the tests are discussed, finalized and documented as a Test Plan in this stage. It is important for all the team members, to brain storm and choose the technique with the smartest test coverage.
Test Case Design: The QA team creates and cross checks clear and comprehensive Test Cases on an atomic level.
Test Environment Setup: Here either QA or the development team fulfills all the hardware and software requirements to deploy a Test Environment and the team tests the major functionalities. One has to be very careful and focused while doing so because this will determine if the software is stable enough for the testing or not. All the actual testing depends on this activity.
Test Execution: The test cases are run according to the given steps and expected results and every test case is marked as pass OR fail and Test Results are created.
Test Closure Activity: Test And Bug Reports are generated and discussed so that the upcoming risks and failures can be handled by using past experiences and knowledge of the current state of the system.
There’s a fact about this model is that it does not have a distinct testing phase. In fact, the testing activities, which in this case are verification and validation, are hooked with each SDLC stage.
Think of it as, whenever there is an artifact produced, you have to create a corresponding test plan or test case document. Checking of the artifacts will also go side by side with this.
But when the actual coding starts, you will execute the test cases that you have created before, and this way the development and testing will become parallel. After each kind of testing, a document with the test results will be produced. The sequence of testing activities will create a testing cycle as follows:
Unit testing: Developers separately check the functionality of each unit i.e. the smallest building block of the whole system.
Integration testing: Different units are combined and tested to see if they are compatible and are well prepared for the system testing and is performed by the testers.
System testing: Tester combine all the modules and test them as a whole. It is to be seen if the system is catering all the needs of the user either functional or non-functional. Performance, reliability, security, load and stress of the system are examined here.
User acceptance testing: The end users tests the system themselves if the delivered system is according to the contract or not.
All the activities will revolve around the goal to keep yourself fully updated with your surrounded team. Your team should maintain such a frequent and clear communication that all of you remain on the same page always.
As you can see that the communication is the key, so, all the stages will involve meetings and discussions. In agile team, QA and Development acts as one. So let’s dive into its stages.
Impact assessment: You will analyze that what areas will be at risk if a change is made in any area. This means that QA team has to find out what areas should be focused if a certain change is induced.
Agile Testing Planning: All stakeholders gather to plan the deployment milestones and testing routines and create a flexible timeline of the project.
Release Readiness: You decide either the current release is mature enough to go live or you have to roll back this stage.
Daily Scrums: Semi-formal daily stand-ups are adopted to coordinate the ongoing tasks of each member and discuss the daily goals and weekly targets.
Test Agility Review: Larger stakeholders gather to evaluate the overall progress. They discuss the risks and improve the processes to mitigate them in the future.
In my opinion, agile testing is more feasible and effective because it allows better risk and change management it is neither too rigid like V-model nor too steady paced like Waterfall. So yeah, it’s cool.
Technology is changing everyday, so is testing. Needless to say, being successful in testing is getting more and more challenging. Being very good at test cases, bugs, test planing, test techniques, etc does not guarantee a success in your testing career.
Don’t get me wrong. I’m not saying that learning the basics and focusing on technical side are no longer important. It’s just that at some point in your journey, you need to stop for a moment and ask these questions: What is the current state of testing? What is the trend? What challenges testing is facing these days?
No, having answers for those questions won’t guarantee a success either but seeing things from high-level perspective helps you understand more about testing industry so you can make the right decision based on the data collected.
In that regard, State of Testing report can help provide you the answers you are looking for about the testing trends, challenges, and characteristics.
So, what is State of Testing report and who’s behind it?
State of testing report is an annual testing report conducted by PractiTest’s QA Intelligence Blog ) in collaboration with TeaTime with Testers testing magazine. This year is the 6th report and they are very consistent with it. The report is also reviewed by committee of testing experts in industry such as Lisa Crispin, Damian Synadinos, Keizo Tatsumi, John Thomas, Anna Roysman & Huib Schoots to make it become a quality report.
State of Testing Report 2019 is not ready to download. I’ll update you here when it’s available to download. Meanwhile, this is the best time for you to contribute to the report, become part of it and provide your influence by taking the survey.
Don’t worry. It’s a very short and comfortable survey (I’ve done it). You can complete it in 5-10 minutes without disclosing any your confidential information.
Are you ready? Let’s take the survey and influence the world (I mean testing world :D)
What’s the most common question from your boss/manager?
I know you have your own answer but I’m pretty sure these are the most common ones:
How much you tested it?
Did you test enough?
I don’t know you or how many years in testing but for me those questions are really challenging.
You can write quality test cases. You can find critical bugs. You can execute test cases pretty fast. But when it comes to measuring your test coverage… it’s a headache.
You don’t want answers like “Yeah, I tested a lot”, “Well, I think I tested it enough”.
You also don’t want to say “I don’t know” either.
As a tester, you not only write test cases, find defects but also provide the information to stakeholders/your boss and in this case you need to be very clear about the your test coverage so that your boss can make decisions accordingly.
In today’s post, I’d like to share with you some ideas to help you measure your test coverage properly.
What is test coverage?
“Test Coverage is a measure, depicting how many possible code execution scenarios, and conditions have been translated into your executed test cases.”
In other words, it’s a metric to tell how much you test something.
Let’s take an example of a new car with a one year warranty and you need to test it.
Can you tell how many years will this car last? Even if you find out the duration, can you be 100% sure about its accuracy?
It is because you don’t have information of the future drivers, weather conditions, maintenance and accidents etc. So you cannot design the test cases of the unknown possibilities.
Now that we know that our test cases will never be mature enough to become 100% sure about our product, so does this mean that we should not test at all?
Of course, not. So how will we determine how much of the software are we testing and how can we know how much testing is already done and how much is remaining?
We will quantify our test coverage by using the given information knowledge and experience. We will create an estimated of 100% coverage as a benchmark, which may or may not be a 100% in actual context. But we will cover as much as we can. There are many ways to do the test coverage which can be used separately or combined depending on the situation.
Here are some of them:
This coverage will result in a white box testing as we will try to create all the conditions by fulfilling which, we can execute either every path, branch, line of the code. With larger application this cannot be feasible enough strategy for a good coverage which does not miss anything.
This one is the most easiest and fastest but this might not be enough most of the times because if the document has something missing you will not be able to cover that part of the system. Secondly, it is very rare in the era of agility, to create a complete and concrete requirement specification because it changes so fast that you can’t risk all your testing efforts on one source.
Here we study the Use-Case document to see what features will be the part of the system and what will not be, and try to clear the boundaries of the system as much as possible and then create test-case based on the use-case steps , preconditions and post conditions. Then we can step out of the comfort zone of that system and check how it behaves, which will lead us to create negative test-cases.
All of these approaches can be used independently or combined according to the software system.
Side-note: Please just use those as reference for your test coverage report. You basically can base on what make sense to your project. For example, you can report the test coverage based on coverage of browsers, OS, platforms, devices, etc.
Once we have decided our approach we can now move forward to create a sort of scale through which we are going to measure our test coverage later because you know, the point of this whole hustle is to derive a value against our testing efforts. Let’s suppose that we have chosen code coverage based test coverage and we have 15 set of functions in our software system out which 12 have been tested to date, then our achieved test coverage can be calculated as :
Achieved Test Coverage = (Number of functions tested / Total functions)*100 = 80%
Similarly for a Requirement based coverage and use case based coverage, if we are using a hybrid we can aggregate the score, or give weightage to each kind of approach and then evaluate that into a single score.
What test coverage is used for?
As I promised earlier, I am going to state how this concept will be helpful to you and how can we use it in a better way.
Team’s Self Evaluation Baseline: It helps us to evaluate our own performance and effectiveness. We can also keep track of our own activities and see how much testing have we done and how much is left.
Quantification of Performed Testing: Our stakeholders will never be convinced about the quality progress until we provide them some numbers and stats. We can achieve it through this.
Testing’s Exit Criteria: Although 100% test coverage never means 100% tested but it helps us in defining some threshold after which we can stop a testing iteration. For example, in a regression testing, we can stop executing tests after 100% test coverage.
Smooth Testing: Our quality team will see the same picture of the current state of the software and its quality. That’s why; testing will be more organized and less obstructive.
Ease in parallel testing and work division: As we have divided our tests strategically, we can easily divide the workload into our team members and they can work parallel.
More grip: Defining the test coverage requires the whole QA team to learn any and everything about the system and hence this gives them more control over the complete testing process and strong base.
Remember that there is always a room for improvement. If you want to do better, then it is really important for you to understand the scope of the system before creating the 100% test coverage criteria. You can’t cover a system if you are unaware of its boundaries. Try to use hybrid approaches to be more thorough in your testing.
So implement and practice this. Explore more coverage strategies and impress your team. And hey! Don’t forget to brag about it in the comments below *wink*
After execution of test case we must analyze the test result and check the defect detected by tool like Test Complete, tosca tricentis, selenium or other. In the first iteration, we can easily show the failed test steps highlighted in “Red” and the pass Test Steps highlighted in “Green”. However, in the next iterations it’s very difficult to do easily the same analyzes and focus the Tester and Developer attention at “first detection” defects. This last category is highlighted in “Red” with the “already known” defect. Is there any tool, addon, or mean that allow to agreggate test results from one iteration to another and make distinction between “first detection” and “already known” defects ? The idea is to have something like this : “first detection” defects highlighted in “Red”, “already known” defects [because detected by automation tool at the previous iteration] highlighted in orange color for example.