Lisa Crispin is a tester who enjoys sharing her experiences and learning from others.In 2012, she was voted by her peers as the Most Influential Agile Testing Professional Person, and given this award at Agile Testing Days.This blog of her provides practical agile testing guidance and contains in depth information on agile testing.
I have a ton of blog posts and webinars on mabl.com, and these are all general articles about testing, DevOps, continuous delivery and test automation, not about the mabl product.
Testing in DevOps Community Site
And I’m grateful to mabl for helping me get a community site to help all of us learn more about testing in DevOps. Wonderful leading practitioners have been contributing guest blog posts every week, and also contributing links to other publications, podcasts, webinars and so on. Please check it out at https://testingInDevOps.org. Please contribute more links to more paces we can learn about continuous delivery, observability, shortening feedback loops, collaborating across roles, anything that helps us succeed at “DevTestOps” practices! And especially, please consider writing a guest blog post for our community. Thanks!
I had really good intentions to blog more about things I learned at European Testing Conference, because I took pretty good sketch notes for myself. But I exceeded my WIP pretty badly between ETC and Booster Conference, and then I broke my leg. I’m making time now to share what I learned from at least one more ETC session!
Romeo Moura explained property-based testing to us. The whole time, I kept thinking, “This sounds just like generative testing, which I learned from my former Pivotal Tracker teammate Ashton Kemmerling. The generative tests found such otherwise-impossible-to-pinpoint bugs with Pivotal Tracker.” I just did some research and found this post from Jason Steinhauser which confirmed my suspicions. That post also has this really good definition from Jessica Kerr:
Property-based tests make statements about the output of your code based on the input, and these statements are verified for many different possible inputs. – Jessica Kerr (@jessitron)
Let’s look at what I learned specifically from Romeo, because it added a lot to my knowledge. He used the bowling kata to illustrate a lot of his points, and I have to admit, I just don’t get the bowling kata completely. But I’m going to try to repeat what he said anyway.
Not just for “modern” code
Romeo emphasized that property-based testing works for legacy code and non-functional programming languages (I’ll say, because on Tracker we used JS). He began by talking about the need to know business rules and examples, and that tests should make the business rules clear.
With parameterized tests, you pass in the examples to a method with an assertion. There is a danger of positives that violate business rules, for example, an invalid bowling score like (3,9). You can’t have more than 7 as the second number in that score.
Property-based tests have valid ranges and generate random values within those ranges. They explain the behavior you are sure about and increase happiness. Our code knows more than almost anyone in the company. Property-based testing asks the code questions, based on our own assumptions.
There are several frameworks for property-based testing (I found some mentioned in this post). But property-based testing isn’t just the frameworks.
A property is something that is always true about an object. it’s ideas about your business. It is different than fuzzing, which is ignorant of your code. (However, you do fuzzing in property-based testing, my understanding is the difference is that the fuzzing is based on these properties that you know about your code, whereas pure fuzzing is totally ignorant of the code.)
Why use it?
The goal of property-based testing is resilience. Thinking back to Pivotal Tracker, we had a single-page JS app with a ton of business logic in the UI. It is designed for some infinite number of users to be able to update the same project tracking information concurrently. The page constantly re-renders to show each user the latest information. That code was written by experienced developers using leading practices such as TDD and pair programming. When you have sophisticated code, you get sophisticated bugs. We had a recurrent problem reported to customer support for months and none of us could reproduce it. A generative test found it within minutes. The developers included some generative tests in the unit test suite for these complicated areas.
Property-based testing doesn’t replace unit testing, it is a complement to it. We like to think we know our code, but if we really knew it perfectly, we’d never have bugs. I am surprised that I don’t hear more people talk about property-based or generative testing. Your team might want to check it out!
I had the luxury of working on a terrific team at a financial services startup from the early 00s to the early 10s. Though we jelled as a self-organizing agile team with expert domain knowledge, we occasionally made some really dumb mistakes when deploying to production. The operations VP, Anne, wagged her finger at us and said we should be using checklists. We took this advice and were happy with the results. I don’t remember anymore if it was Anne who recommended it, but I finally read Atul Gawande’s The Checklist Manifesto, which totally sold me on the value of checklists. Even when you think you couldn’t possibly forget some obvious items on your checklist – you can! Just use the checklist!
I recently had an unhappy accident where I slipped on hidden ice in the donkey pen and broke my leg. This necessitated surgery. I was fascinated at how pervasive checklists really are at hospitals. Medical errors the third-leading cause of death in the U.S. I was quite happy to be asked repeatedly for my name and date of birth (DOB), have my hospital ID scanned, repeat why I was there, what was my injury, what was I having surgery for, which leg was it. I was glad to see the kind and comforting hospital staff record every bit of information and sanitize their hands over and over.
Even the anesthesiologist quizzed me as to my specific injury and surgery. He then gave me options for anesthesia. I could have spinal and be awake or semi-awake, and recover faster from the anesthesia. Or have general anesthesia. I asked about a nerve block because someone else had mentioned it. This brought a visit from the pain management team. As I made decisions, they were noted down on the consent form and we all signed. Which leg are we operating on? What surgery is being done? Repeating the plan for anesthesia and pain management over and over. Initialing changes as they were thought up. It was a whole team approach – the surgeon, the main anesthesiologist, the pain management team all consulted each other as well as me, the “customer”. Final decisions were up to me. What is your full name and DOB?
Having a spinal anesthesia meant I was awake in the operating room when the team went over its final checklist. What are we doing? Which leg is it? The surgeon signed my big toe and my thigh on my right leg with a Sharpie. Yes, it can seem a little silly, but think of the impact of doing the wrong leg or the wrong person!
There was a list of things to check to make sure the anesthesia was wearing off, a list to make sure the nerve block was done in the correct location (though I have to say, I never felt it had an effect, I had plenty of pain!), a list to hand me over to recovery. Taking my vitals every few hours. What’s your full name, what’s your DOB, what did you have done? They obviously kept careful track of my pain meds and gave me the options available to me at any time I asked for more.
Boring is better than disaster
I imagine going over the same checklists hundreds of times a day must be boring for the hospital staff, though they can record the answers digitally these days. If it prevents even one death, it’s worth doing, and I suspect it prevents a lot of mistakes. Let’s embrace a bit of tediousness and mitigate the risk when it’s essential to do so. Having all that covered will free parts of our brain up for creative problem-solving and decision-making. We’ll have clear communication to deliver valuable changes to our customers. We’re able to automate so many of our checks, but we may need to check in on a human level too.
Continuous delivery is meant to be boring! The only excitement we want is customers enjoying new features when we turn them on in production. In a way, our pipeline is a checklist. We check off each stage. We do it over and over. We keep improving it. We make sure we’re working with the right people and the right things. We prevent silly mistakes. Sometimes you can’t cut corners. It may be boring and tedious, but it is easy and the payoff is big.
I had the pleasure to participate in Booster Conference in Bergen, Norway, aptly tagged “the conference for the whole team”. Many of the highly original and engaging sessions are on video and available for free on the Program page, go check them out! Making all that content freely available is just one indication of how amazing the volunteers who put Booster on are.
I never got any more posts up about European Testing Conference so far, so realistically, despite my good intentions, I may not find time to write more about Booster. But I have my sketch notes from both, hope springs eternal. I learned so much at both conferences.
In this post, I want to summarize a few key points I learned from Linda Rising in her workshop on influence as well as her fabulous keynote. I owe a lot of my own success to all the things I’ve learned from Linda over the years. The patterns in her book with Mary Lynn Manns, More Fearless Change, helped me through a lot of struggles to change culture and improve ways of working on my teams over the years. The idea of doing small, frugal experiments to continually learn and improve was transformative for me. Interestingly, I am often mistaken for Linda, I have no idea why, except that perhaps to the younger people out there, two women of a certain age with short hair and a name beginning in “Li” look the same? I don’t mind though, I’d love to be as awesome as Linda!
Tips on learning
Linda started out her workshop by making sure we all had something to drink in front of us, as well as paper and pen to take notes or doodle. “Look! The water is free! The paper and pens are FREE!” she encouraged. She also urged us to get up and move. She herself always stands and walks around when attending someone else’s session. These are all techniques based on the science of how our brains work. We learn better when we move, use our hands, take care of our bodies. Ashley Hunsberger and I borrowed this great idea for our own workshop at BoosterConf, and I will use it going forward!
The bottom line on influence
Linda Rising at BoosterConf 2019
We all run into resistance, both professionally and otherwise. You’ve probably wanted to get your team to try something new, and you couldn’t convince them. There are good reasons for that! But don’t despair, there is hope. Here are the main ideas I took away from Linda’s sessions.
In both her workshop and keynote (the video is up, please watch!) Linda explained how our brains are hard-wired with certain biases and behavior that is rooted in our evolution as humans. We make decisions based on our own values, and you can’t pretend to have someone else’s values – they are values. It is kind of impossible to change people. And forget all that logical thinking, logic will not help you persuade even the most intelligent person.
That said, there are patterns we can try which may help us get people to try new things and collaborate with us.
Ask for help
The “Fear Less” pattern says we’re afraid of people who disagree with us. Linda says we should learn from those people instead. She gave us several patterns to help.
Linda told a story about how Ben Franklin got an enemy on his side. Franklin needed to win this person over in the Constitutional Congress. At the time,
Ben Franklin, Turn Enemies into Friends
books were incredibly rare, treasured and valuable. Franklin asked the man to loan him a particular book from his collection. Probably that man was surprised, but he did loan Franklin the book. The thought process that goes on here is something like, “I really hate that Ben Franklin. But I loaned him my valuable book. Maybe I do like him?” They became lifelong friends. Linda called this “commitment and consistency”. It’s why companies want to give us free trials and little gifts. (You can easily find this story online).
You don’t have to ask your “enemy” to loan you a book. You can simply ask them for help. Ask for advice, ask for a favor. Linda quoted Arthur Helps: “We all admire the wisdom of people who come to us for advice”.
In her keynote, Linda told a story of dealing with a sceptic who wanted to block the adoption of agile principles and practices in a company where she was consulting. She could not find a way to win this fellow’s cooperation. One day she encountered him in the hall and he started ranting about all the things that were terrible about the transition to agile. She was sort of trapped anyway so she just let him talk. Eventually, he kind of ran down and said something like, “You know, maybe it could be a good thing.” She listened him into agreeing with her.
The “Listen, listen, listen” pattern says to use silence, short responses, body language, questions to promote discovery to show you are working with the speaker. It sometimes works. Maybe nobody else is listening tot that person. We all want people to listen to us! And you never know – if you can overcome your own unconscious biases, you might learn something from that person!
I’m always trying to improve my listening skills, but I’m such a talker, and I always fall into the trap of thinking logically. There’s no reason to waste that energy. Instead I will put my energy into listening and learning.
I’m fizzing with all the stuff I learned at European Testing Conference in beautiful Valencia last week! I gave myself the gift of taking sketch notes of all the sessions I attended, so I’m hoping I can do a few blog posts about it. I have been neglecting this blog lately because I spend so much time writing posts on mabl.com. (Which I hope you will check out – they are not about mabl!)
Anyway – I’m gonna start with the very first session, the keynote by the awesome Angie Jones. It was the perfect kick-off for two days of inspiration.
A surprising turn of events
Angie Jones describes her “10 Ps” report card
Angie has helped us get traction on the road to successful test automation for a few years now. She has given our software community the amazing gift of Test Automation University, which is free and incredibly valuable – I’m enjoying the courses myself.
When Angie started describing her approach to a huge automation challenge when she was working at Twitter, I expected to learn that she came up with a creative, successful solution. In order to do the testing she felt was needed, she automated creation of test data and setting up scenarios for further testing. Lots of head nods and “aha moments” from the audience as she told her story. (I’m writing this from my notes, so I hope I’m recounting this accurately enough. I believe there will be a recording available of Angie’s talk.)
Angie recounted how she found she needed an API endpoint to retrieve data to help with testing, in an area of the app that “belonged” to another team. Unfortunately, that team responded that they did not have time to do that work, even though it wouldn’t take long. Angie sought help from her manager, who arranged for three teams to get together to discuss the request. Whole team approach to testing, right? That sounded like a great way to get the problem solved. However, the result of the meeting was that a decision that there was indeed “no time” to create the endpoint. Angie hit the proverbial brick wall. It’s happened to a lot of us – but hard to believe it could happen to Angie!
Testability is a team responsibility
Rob Meaney’s 10 Ps of Testability
Angie emphasized testability should be a requirement of the feature you’re developing. She referenced Rob Meaney‘s 10 Ps of Testability model. (I’m looking forward to Rob’s upcoming book on testability with Ash Winter. If you’re a Ministry of Testing Dojo Pro member, watch his master class on testability.) She showed how many of the “P’s” were missing for her team in this situation. It’s hard to understand why any organization would fail to make good choices that let them build quality in. Unfortunately, some simply don’t understand the value of investing in quality.
It was surprising to hear Angie describe anything less than a success story. What is so valuable about this story were her many lessons learned. We can all learn from her story and potentially find ways to get teams engaged in creating testable software.
True leaders like Angie aren’t afraid to share failure stories as well as success stories. As I attended subsequent sessions at the conference, I’d learn about a new idea, and think about the ways it could help with the scenario Angie described. Angie had such creative ideas for testing in production, automatically generating test users, automatically updating a field in the database to facilitate the testing without affecting actual customers. If she could have gotten one more bit of help from teams outside her own, she’d have been able to easily check whether ads appeared to the right targeted users.
I felt that Angie’s larger message was that a whole team approach to building quality (including testability) in, with support for automation that enables the team to succeed with testing, is key. When the whole team works together to try new experiments and see what helps them progress their quality goals, the magic happens.
Valuable lessons learned
I’ve been extremely lucky in my career to mostly work on teams that were allowed to take time to learn and solve problems, including finding good ways to support testing and quality with automation.I have lots of success stories to tell. But as with Angie’s keynote, maybe the failure stories are eve more powerful.
I’ve also worked on teams where the culture worked against collaboration across all teams and roles within teams. Even when I led a successful agile project within the company, people were unable to let go of the way they were used to working. That way wasn’t working, the product was failing as a result, but I think they were so stressed out they just clung to something familiar. I used patterns from More Fearless Change that usually work for me, but I failed in that organization. I bailed for an opportunity to work on a self-organizing Scrum team, where our whole team worked step-by-step to succeed with test automation. The lessons I had learned from my “failure” helped me contribute more in my new team.
We’re always hearing how failing is good and we shouldn’t be afraid to experiment. However, in some company cultures, it isn’t safe to fail. If you’re working in that type of environment, see if you can be an agent for change. And if you can’t get any change going, see if you can find a better opportunity. Whatever happens, don’t beat yourself up for failures. Our most appreciated leaders also fail sometimes. You’ll be a better team member in the future if you can see a failure as learning from an experiment.
I’m not much on predicting the future, but I shared my thoughts on current trends in software testing and testing careers, along with some other leading practitioners (including Janet Gregory) on abstracta.us’s blog.
Janet and I will be doing our “Agile Testing Essentials” one-day tutorial at TestingUY in Montivideo this coming May! Please propose your own session, we’d love to see you. Or just buy a ticket!
Joel Montvelisky and I paired to talk about how testing has changed in light of more teams adopting agile development and DevOps culture. Many thanks to mabl and PractiTest for collaborating to host this! We had a great chat about how we see the role of testers and test/QA managers, and the ways they add value to teams changing in today’s continuous delivery world. We talked about Alan Page and Brent Jensen’s Modern Testing principles. We also discussed the DevTestOps manifesto. Our participants asked a couple of great questions as well.
Janet Gregory and I talk about testing, DevOps, DevTestOps and more with the awesome Cucumber team on this episode. I feel so honored to have the opportunity to participate on one of my favorite podcasts!
I was honored recently to be interviewed by Phil Burgess for his IT Energizer Podcast. Check out my episode. What career advice would you give? I’ve enjoyed many episodes, so many different insights and stories. Check them all out!
It’s been a big year of change for me. My husband and I moved our three donkeys, three cats and two dogs 1900 miles, from Colorado to Vermont. I started a new job as testing advocate for mabl at the beginning of August, and executed our long-planned move three weeks later. We love our new home, quiet and beautiful yet close to everything we could want. We’ve made many new friends.
I’m extremely fortunate to get to work from home most of the time. It’s a relief to no longer commute more than an hour each way to work three days a week (I worked from home two days a week in my previous job). I get to be a country girl and hang out with my donkeys. I work with a great team of smart, passionate people, building an exciting product.
The rocky road of working remotely
Perfect, right? Well, it’s pretty awesome. But there’s a downside for me. I’d much rather work in an office where my teammates are physically present. I depend on continual feedback to know if I’m contributing the right value for my team and for our customers.
My team makes the extra effort to include me in meetings via video, and conversations go on all day in Slack. I can sit in on video calls with customers and prospective customers to learn about their testing challenges. My main job is helping people learn ways to overcome their testing challenges and continually build their organization’s quality culture. I do that mostly by writing, speaking, webinars and the like. I also participate in online and local communities of practitioners.
And yet, I often have the feeling that I’m working in a vacuum. I can get feedback, such as having someone review a blog post or brainstorm ideas for a webinar. But it’s not the same as asking the person sitting next to me if I can show them something on my screen, or getting a product owner and a developer together for a quick discussion about a feature. I find it difficult to work with a longer feedback loop than I’m used to.
Frantic for feedback
I confess that I’m something of an affirmation junkie. I love it when I come up with a question that makes someone say “Oh, great catch! We hadn’t thought that through.” It’s so rewarding to pair with someone and see them get excited about learning a new testing technique. Working remotely means I get fewer of those “high five moments”.
In a feedback void, I tend to start listening to my inner critic. My imposter syndrome fires up big-time. “Nobody commented on my latest blog post. Either they don’t even bother to read it, or they think it sucks.” “I reported that bug I found in the Slack bug channel, but nobody responded. Do they just think I’m an idiot using the product incorrectly? Or are they just ignoring me?” “I asked for help with getting analytics on my webinars, but nobody offered to help, they don’t care.”
This is my own doing, not on my teammates – my brain is totally making that stuff up. But sometimes, I can’t help it. I know I need to “keep it real” and get my confidence back. I know many techniques to banish my inner critic. I know I shouldn’t compare myself to other people, but I see people I admire who have a similar job to mine who appear fearless and full of amazing ideas – with the ability to implement them.
Reflecting on my self-doubt has made me realize that my accomplishments have come from being part of a team. I can collaborate with other people to come up with great ideas and deliver on them. When I try to work solo, the ideas don’t come, and I lack the mojo to try something daring. I lose my focus, and spend time on things that aren’t the best way for me to contribute value to my company and my community.
Asking for help
I work with terrific people. I need the courage to ask them for more help. With their help, I’ll be able to help more people in turn. I’m planning a trip to Boston so I can have some face time in the office. Not asking for help would lead to disaster, so I’m going to do the smart thing. Collaborating with colleagues, we’ll be able to try experiments and learn how I can best contribute.
This has also made me realize that feedback from the software community is a huge benefit for me. It’s so valuable when a workshop participant makes suggestions how I could make the workshop better. When comments show a blog post resonates with readers, I know that’s a topic I should explore further.
Dear readers, I’d love your help too. Those of you who work remotely, how do you engage with busy teammates who are colocated in the office? How do you get them to pair with you over video, especially if they aren’t used to pairing? What do you do when you can’t hear the people in the conference room in the video meeting? I need tips and tricks to really feel part of the team even when I’m 250 miles away. How can I get that fast feedback that I rely on?
I’d also love to know ways I can help you, since that’s really my job. I’m backed up by a company that wants to reach out to the community of people who are passionate about software quality and testing. I want to use my powers for good, and I need your suggestions to do that!