We think we know what it feels like when we do a good job
- Article tags
To deliver meaningful results as a pentester you have to be both patient and persistent. You have to love the process and strive for results for your clients. You also have to go in-depth and cultivate a broader understanding of all the pieces of the puzzle.
Today’s guest, Willa Riggins, talks about how “every small piece contributes to the larger picture” in pentesting and explains why “it's about understanding the intricacies and appreciating the craftsmanship."
From the mindset behind excellent pentesting work to the (difficult) things that are never going to change in this space, we glide through Willa’s experiences, hard-earned know-how, and thoughtful approach.
Willa bio
Through her role as a manager and community leader, she emphasizes the importance of building a team that resonates with strong values and that’s willing to cultivate both technical and “soft” skills to do meaningful work.
Willa talked at various infosec conferences such as DEF CON, BSides Orlando, Florida Cyber Alliance’s CyberCamp, and several Central Florida information security groups such as OWASP Orlando, DC407, HackUCF, and OrlandoDevs, generously sharing her experience with those looking to build theirs.
Dive straight into the convo to learn:
Why you need to get comfortable with trial and error to enjoy pentesting [03:43]
The key lesson Willa learned from working in app security [09:45]
How to focus on your craft when reporting vulnerabilities [13:14]
The challenges pentest teams face in making their work count [19:07]
The realistic, reasonable way to use automation in pentesting [24:28]
Two aspects of the hacker mindset worth cultivating [28:36]
Why (and how) having a hobby outside pentesting makes you more productive [33:33]
How to set realistic expectations around developing a career in the field [36:42]
What will be the key differentiating factor in penetration testing [42:40]
We believe you’ll get wisdom and inspiration from this kind and generous conversation. Willa will help you get a broader understanding of this field highlighting the fundamental role of people and teamwork.
Just hit play!
Resources from this episode:
Willa’s personal website
Willa on LinkedIn
Willa’s talk on esoteric exfiltration at Def CON 24
Women in Locksport series: Interview with Willa
Wicked6 Cyber games
Listen to this episode on:
Episode transcript
Andra Zaharia: Only the most curious and persistent people thrive in offensive security. How do I become a better hacker? How can I build and maintain my advantage over adversaries?
And what's limiting my ability to think creatively?
This podcast is for you. If you're the kind who's always digging deeper for answers, join me as I talk to some of the world's best offensive security pros about their work ethic, thinking, and real-world experiences.
This is We think we know, a podcast from Pentest-Tools.com
[00:42] Andra Zaharia: Sometimes penetration testing is boxed into a narrow, shallow definition which is neither real nor fair to you, the person doing this type of challenging work. To deliver meaningful results as a pentester, you have to be both patient and persistent. You need to both love the process and strive for results. And you need to go in-depth but also cultivate a broad understanding of the elements at play. It's a fine balancing act. It's a craft. It is an art form.
And today we're lucky to have a guest that not only proves this through her work but also teaches it to others through her role as a manager and community leader.
Willa Riggins talks about how every small piece contributes to the larger picture in pentesting and explains why it's about understanding the intricacies and appreciating the craftsmanship.
Examples from her over 15 years of hands-on experience across application development, offensive security, and management highlight the fundamental role of people, the people who make technology, the people who test it security, and the companies that keep it safe to use.
Andra Zaharia: From the mindset behind excellent pentesting work to the things that are never going to change in this space, we go through Willa's experiences hard-earned know-how and thoughtful approach. This conversation is a treat, and it is a privilege to share it with you.
Willa, I'm so excited to talk to you today, and I'm so grateful to have this opportunity because you are not just a super generous and supportive person of other people, but you bring such wonderful experience from almost every angle imaginable in this space, and I cannot wait to impact your stories for people to learn from. So thank you so much for being here.
Willa Riggins: Well, thank you so much, and no pressure on me. I am glad that a good reputation precedes me at least.
Andra Zaharia: Well, absolutely. And it is me who I feel the pressure in the sense of trying to do right by your knowledge and your experience and your thoughtful insights. And something that I wanted to actually start with. You have a lot of experience behind you as an individual contributor, but also as a manager for a large team handling really complex projects.
So you have this great big-picture view that gives you insight not into just how you do work individually, but how that dynamic changes and evolves over time. So I was wondering, what realities of the job did you bump into when you started in pentesting, and how have they changed over the last ten years?
[03:43] Willa Riggins: Oh, gosh, that's a big question. Early on when I started, I think every pentester goes through this. It's the excitement of the job. You think every day is going to be “hack all the things, get all the shells and exfiL all the data”, and it's not like that at all.
I think I was explaining to one of my direct reports the other day, a lot of what we do is research and try research and try. And nine times out of ten, what we're trying is not going to work. So it's a lot of kind of trying to solve a puzzle that might not have a solution. And I always coach folks who are trying to go into the industry that if you like solving problems, you might like this job, but you also have to like solving problems that might not have an answer.
Willa Riggins: Sometimes the answer is that there's just not a flaw here. There's nothing to find.
And that can be really difficult, especially if you're doing that every day, day in and day out.
That impostor syndrome, that lack of dopamine from finding the thing can happen really easily.
But also there's more to the job than just finding flaws. Sometimes that means writing a report, and sometimes when you deliver that report, you have to explain what you found. And there's a lot of kind of talking about empathy and working with stakeholders and developers and whoever it is that's receiving that finding to explain why this is important.
So there's a lot of storytelling in it as well, but just there's a lot of different dimensions of what we call penetration testing and pentesting and red teaming and all those different pieces that really make up the job. It's not just having really good skills, it's also being able to translate those into what a business needs.
Andra Zaharia: It really is, and I love that this is a topic that's getting more attention and more intention from people in this industry because they see how it makes a difference, and they see that the people we look up to in the industry actually have spent time developing these particular skills, and they've not only helped elevate the community by itself, but it also benefited their careers, of course, which is obviously what we want people to experience in their lifetime.
And talking like building on this particular idea of this is something that's always been important and will most likely continue to be extremely important no matter how fancy and smart tools get because there are so many other aspects that tools cannot even begin to cover. What drove you at those, when you were kind of facing the challenges that people in your team face today?
What kept you going? What kind of fueled you when that lack of dopamine, when those repetitive things started to kick in and start to be a bit disconcerting?
Willa Riggins: Sure. I think a lot of it is kind of retuning your brain to think in terms of we're not just proving negatives. We also want to talk about positives. We want to talk about what teams are doing well.
So when we don't find cross-site scripting in a web application, across the entire application, what did they do right? What did they do to mitigate that vulnerability?
Same with SQL injection, all the different OWASP Top 10. What is that team doing right? Or maybe it's a process thing. Maybe we were doing social engineering. Maybe we were doing some kind of other activity that they were able to prevent that through process.
Or maybe their people are just really good at spotting things. It's good to call those out and not just look at this is what I found on my test. These are the things you're doing wrong.
This is how I think you can improve. We also need to talk about what are they doing right, what are the things that are different, but also understanding and being self-aware that I'm not always going to find every finding.
[07:45] Willa Riggins: We're human. That's what we do. Every person's different. Every person's going to find different findings. We like to call it - when I was in consulting, it's like it's a point in time assessment done by a human being. We're not going to find everything. It's not an exhaustive test as much as we'll try to be comprehensive.
Every pentest is time-boxed. If we had unlimited time and unlimited labor on a pentest, it would be awesome. We'd probably find so many more things. But I think it's really learning that.
And you learn that kind of through doing the job for a while and learning that, oh, this is what this is like. This is what it feels like when I think I've done enough. This is what it feels like when I've done a good job and kind of building that self-confidence and that knowledge that I've covered all the bases, I've done all the things I think I need to do, there will still be mistakes, and that's okay. I think that's the big piece. And it really comes with seniority and kind of having spent time on the keyboard and really doing the work.
Andra Zaharia: It absolutely does. And thank you for highlighting that expectation of an ideal scenario where you'd have all of the time and all of the resources, all of the things when that never happens, that would produce all of the findings, but who would take care of them? Because you need to have a little bit of resources to actually tackle all of those findings and make them happen and apply order remediation and follow all the guidance.
And I was wondering if you have from your previous experience while working on application security, or even further back when you were working in app development and doing sysadmin stuff. What did it feel like when you were on the receiving end of pentesting findings and what that experience taught you or how it served you when you actually became a pentester?
[09:45] Willa Riggins: Yeah, absolutely. I think, unfortunately, my development career started really early, before application security really took off. So really my kind of first experience with application security was being on the other side. So switching sides from being a developer to then giving all the insider knowledge on how bad the code base was to our leadership in helping develop the first application security program at that company.
But what I kind of learned is generally developers want to do the right thing. We all want to have perfect code. We don't want to have vulnerabilities. Nobody's looking to do that.
But the challenges are really in what we mentioned earlier was resources. Do I have the time and the money to go and fix these vulnerabilities? Because I want to. I want to have impenetrable code. I want to have the best application I possibly can, but can I afford to do that?
And that's what I learned early in my career - is very much that everybody wants to do the right thing, but they're not always able to.
Willa Riggins: And now in my current role, kind of learning what risk management looks like, how do we decide what to fix and what is okay, or what can we mitigate in another way?
And that's really kind of taught me that even though everyone wants to do the right thing, we can't do all of them. So let's take a risk-based approach to where we spend our time and our money, both on the pentesting side to find flaws, but also on the risk remediation side.
We're not going to spend the same amount of time fixing a low as we would, maybe a critical or high, but it's important to kind of align those resources where we think they're going to do the most good.
Andra Zaharia: And that is such an important aspect of the craft and such a deeply human kind of ability to be able to take all of that context information, tie it to the business context, understand where it makes sense, understand what makes sense specifically for that business in that sector with those particular compliance restraints and so on and so forth. There's just so much that goes into making these decisions. And has this decision-making process changed what you expect of the pentesters that you manage?
Willa Riggins: I think kind of, I like to think of it in kind of two different parts of how we measure risk and impact. One is kind of our technical risk. This is in a vacuum without all the modifiers for what industry we're in, what our compliance needs are. What is that finding in a vacuum? The CVSS score of a CVE that's out on the Internet. What is that score? What does it mean?
And then kind of taking that and looking at what we call a residual risk, what does it look like for us? And that can be very different. Maybe there are mitigating factors like network security, firewalls, detections, web application firewalls. What are those mitigating technologies?
And then is it on the perimeter? Is it all on the Internet or is it just an internal application or who uses it, how many users are on it, what data does it process? All those different things, kind of then modify that residual risk down to understand where does it actually land for us, what's most important?
[13:14] Willa Riggins: So I think my ask of my testers is to keep reporting all the things. Don't let the environment affect how you feel about a vulnerability. Do your best to be objective, to measure it in a vacuum, and then apply kind of what the business thinks about that vulnerability. And we work really closely with our stakeholders in product security as well to kind of calibrate some of those, because we don't always know that. As pentesters, we're not experts on every domain and every business unit in a company. We only know what we are given and what we know from our experience of being there. So oftentimes partnering with other parts of an organization or even in a consulting environment, partnering with the customer to understand what are your needs and what makes the most sense for you.
But I think my ask of testers is just keep doing what you do and do it well because we get into this kind of, we don't want to commoditize pentesting. We don't want to make it where it's just we're checking a box. We want to continue to do the high-level, in-depth work that we're supposed to do as part of our craft.
Andra Zaharia: I love that. And that distinction is so helpful because yes, there is this - more senior people in the space are definitely trying to help younger pentesters develop that ability to think in context, to connect things to the business needs to be able to walk business people and decision-makers with less of a technical background through their report in a way that makes sense for them and that persuades them to make all of these decisions.
But how you build that argument for yourself and what that process looks like on your end of things can be different to this. So they don't have to necessarily come together, they can come at different stages. So thank you for making that distinction. And I was wondering if we might look at a quick example because at a quick example of a particular vulnerability, we can pick any of them because the apocalypse sound like twice a month in penetration testing.
Andra Zaharia: So for a situation like Log4Shell, for instance, what does that look like from your perspective as a manager leading a team of penetration testers and having to cope with this kind of vulnerability that everyone's raging about, what do you do first? How do you approach that in a way that's mindful of all of the things that we've talked about so far?
Willa Riggins: Sure. I think one of the biggest things for us is really confirming exploitation because we have folks in application security and across our development teams who can look at kind of the composition of an application, tell you if Log4Shell is going to impact their application, but what they can't tell you is can you exploit it?
And I think that's really where pentesting comes in and really adds a lot of value there is to say, hey, we know this is out on the perimeter, it's going to take us x number of days to fix. Can you exploit it? Is this something we need to escalate? Do we need to do it today?
Willa Riggins: And so that's where my team will come in and kind of do their poking and prodding and see just how hard is it to exploit this, to hopefully give our defenders a little bit more data to work on so they can prioritize. Because at that point every Log4Shell entrance is important, right? Everything is urgent, it's critical. We got to fix it right now. So how do you kind of split hairs? Where do we start fixing things? Where should we go first?
And so that's where my team generally kind of helps out with things like Log4Shell is to say yes, we can exploit this. This is how easy it is. Here's a PoC that you can use to test your remediations or your detections if that's the way that we're going as well. But really our part is to help our defenders, help our developers to get to a remediated state, whether that's testing what they've fixed to see: Hey, did your fix work or testing for exploitation to tell our defenders - one, do your detections work? And two, how easy is this so that we can kind of feed that into whether it's a CVSS score or the vulnerability management system in some other way? And that's really kind of where my team has helped out a lot with things like that, any kind of zero-day situations.
Andra Zaharia: Because prioritization seems to be not just a keyword for our work but for our lives in general. I feel like penetration testing itself makes you so mindful of let's say constraints and of limited time and what you can achieve in that time and using constraints actually makes us more creative.
That's like a documented fact and many people use this to their advantage while cramming to study for a university exam among other things. But that practice itself, I feel it's such a test of maturity because as you evolve you get better at spotting these things, you get better at developing patterns and processes and mindset and ways to approach things.
But what particularly stands out about how you talk about these things is that you seem to have helped your team cultivate a very good relationship with developers and other teams in the company.
And I was wondering if were there any particular things that you did in this direction to make sure that communication is smooth and make sure that people are working together and it doesn't feel like an adversarial relationship where pentesters are finger-pointing and making the developers and defenders feel less ideal about themselves.
[19:06] Willa Riggins: I'll say upfront, it's a work in progress. I think any offensive security team is going to struggle with that same problem. I think what my team has done in the last couple of years is really try to meet developers where they are.
So first off, when we report findings we have a readout. Most consultants do that too. But when we have our readouts we really want to discuss the vulnerability. Does it impact it? Who owns it? Is it the development team? Do they own the infrastructure that we're reporting the vulnerability against? Or is it an application vulnerability?
So first kind of understanding ownership so that we're not catching them off guard because if it's something they can't fix, we're not really enabling them to go and resolve the issue and then second explaining things to them. So we have twice a week my team does office hours and that's really to allow developers to drop in and ask questions.
Willa Riggins: So if they get a report and they're like, I don't understand where this problem is, or I don't understand how to fix this, they can drop into our office hours, have a question, and we partner with our product security team as well. So if they have detailed remediation needs that are very specific to their application, they can ask those folks too.
And that really enables that kind of two-way communication that is less kind of ad-hoc because we're used to getting emails and instant messages and all those different things. And it's really hard for a team to stay focused during that, both for the developers and for the pentest team. So we've kind of funneled that activity into those office hours and also kind of a shared mailbox. But the idea is really to give them access to someone, whoever it is, on the team at any given time, so that when they ask a question, they get a prompt answer. And that's really helped, I think.
The other thing we've done is we all know about - when you write your pentest report, you publish your findings. Those folks have x number of days to go fix them. And so our organization decided, hey, we're going to lower that. We're going to lower that threshold and try to get those to fix faster.
Willa Riggins: But then as a pentest team, we can't take longer to do our assessments and do our retesting. We have to lower that as well. So we actually lowered our retesting turnaround time significantly to allow the developers more time to spend on their remediations. And that's really helped kind of build that culture of we're in this together. I think kind of our next step, the thing that we're looking to do going forward is to really integrate with their SDLC process - working with them to integrate into their agile pipelines, really kind of work inside their project so they can plan for the pentest as part of their work, as part of their validation.
So it's not a surprise because a lot of these teams get caught off guard. They have to do their compliance testing every year, and every year it's still a surprise because they don't know when it's going to happen and we're not working closely enough with the people doing the work to really make an impact there. And so that's something we're doing kind of going forward. But again, always a work in progress.
Willa Riggins: A lot of folks look at pentesting as a compliance function, and I think that kind of downplays what we really do. This is very creative work, very sideways thinking, and sometimes we find things that they're not compliance-related, but they are incredibly important. Maybe it's business logic, exploiting a process flow, something like that.
But trying to retrain folks that we're not just a compliance function. We're here to help you do the best, that you make the best applications and the best infrastructure that you can.
[22:45] Andra Zaharia: Thank you for sharing, that's such a thoughtful approach and thank you for sharing the details of that. I feel like that's very generous and helpful for other people to understand how these things can happen and how small changes can start improving those relationships. At the end of the day, that's what makes the company work. That's what keeps the company safe.
Tools and processes may be nice, but without those relationships to actually make them happen, that's just going to be nice words on a piece of, well, I wanted to say paper, but nowadays, thankfully we discarded some of that.
Andra Zaharia: You talked a lot about some aspects that I feel are so important because pentesting does get boxed in into this very narrow, sometimes superficial definition that becomes frustrating to the people who do this work because their work gets misinterpreted and mislabeled and honestly not appreciated.
So when you see someone come around and claim that that can automate your work on top of all of these things that already make you feel like I'm not getting enough recognition, I'm not being seen or valued according to the work that I do, and then this comes along that can really hit a nerve. So I was wondering, how do you talk about automation in your team? How do you relate to it and how do you obviously use it to make your work better, but without having this generalization that honestly helps no one and is unrealistic as well?
[24:28] Willa Riggins: Yeah, I really think when we talk about automation, we have to kind of frame what that means because a lot of vendors and companies with that silver bullet automation are looking to replace the human. I don't think that's the path forward. I think data fusion looking at how do we use automation to help our people make better decisions, that's where we need to spend the time, and we're using that today.
When you look at products like BurpSuite, BurpSuite tells you, hey, I think there's a vulnerability here. It can't tell you definitively, but it'll say, hey, I think there's a cross-site scripting attack here, or hey, I think there's a SQL injection here, or this library might be vulnerable. But it really takes a human being to kind of put that together, chain those vulnerabilities together into an exploit path, an attack path. And I think that's where really the value out of automation is.
Willa Riggins: But when we think about it a little more broadly, a lot of companies are doing static code analysis. They're doing dynamic analysis. They have a lot of data. And so building those feedback loops to take some of that data and take a look at it and help the pentesters kind of target where they're going to spend their time.
Because if we have a lot of cross-site scripting or SQL injection, maybe we can do something with that. Or maybe we need to spend more time on the business logic, because those different tools don't cover that. They don't have that insight.
And I think that's really how we need to frame automation - it's more of kind of giving the human the data that they need to make better decisions versus replacing them entirely. And while I do think there might be some time savings, I don't think it's as much as people think it is. I really don't.
I know a lot of companies will sell kind of, oh, we'll do a scan and a test, and I say, well, what is that percentage? And a lot of places will tell you it's 80-20, 80% manual, 20% hands-on scanning, and trying to work through that and really understand, what does that mean?
Willa Riggins: I think kind of is extremely limiting for testers because a lot of the tools that we use, a lot of the things that we use to do our job are automation. But what puts it together is a human being.
Andra Zaharia: Oh, that's absolutely, you framed it beautifully, and I feel like you brought such clarity to a topic that gets so murky depending on whose interests are at stake. But in the end of the day, I feel like respecting each other's insights and experience and contribution is the most important thing that gets us to learn the most, that gets us to create things that are actually valuable, whether it's technology or processes or training or whatever it is, and to actually stay firmly rooted into the reality of things instead of trying to portray, which we sometimes do in this industry. The space is such an obscure, very cyberpunk vibe kind of thing.
And then people come into the industry for the hype of it, and they might be slightly disappointed about the sometimes mundane reality of things, of getting things done in this space. So I really appreciate you bringing such a mature and balanced approach to this space. And speaking about targeting your time, especially when it comes to learning versus doing, exploring all of these things. There's such a vast amount of possibility in this space.
Andra Zaharia: How do you help the people on your team and the people who you mentor find a way to combine focus with also having the flexibility to explore things? Because that is definitely one of the biggest challenges that I've seen everyone struggle with, including myself, although I don't have necessarily a technical profile. But the learning, of course, never ends in either direction as well. So how do you help them with insights from your experience of doing the same?
[28:36] Willa Riggins: Sure, I'm fortunate. I have a great team. They're all continuous learners. They all want to keep doing what they do, and that's really helpful is to build the right team. Obviously, hindsight is 2020. You don't always get to know who people are before you hire them. But been very fortunate there.
I think what I've done as a leader is try to make time for that, to set aside the time for that continuous learning because if we don't learn on the job, we're never going to keep up with the industry. So we've built a culture where continuous learning is encouraged and we make time for it during working hours.
Willa Riggins: We have, I think, two half days every other week where we spend some time kind of, I guess you call it sharpening the saw and really working together and understanding. We do CTFs together as a team. Every once in a while I'll even try to participate. It's been a while since I've been hands-on keyboard, but still got a little bit of chops here and there, but really sharing knowledge across the team without looking at titles, we just all kind of pitch in and look at what everyone's doing and share.
And that's really kind of how we do things on my team is just being intentional about spending time to learn. But also on our tests, when we were doing our penetration testing, we also earmark things for this need more research, obviously. Like I said before, we can't spend all the time in the world on one engagement, but we can kind of say, this is interesting.
Willa Riggins: I wonder if this is anywhere else and kind of make a note of that to go look for that or pass it off to our application security team to hey, maybe you need to dig a little deeper on this one. And that's kind of how we manage that on my current team, but also encouraging folks to submit to conferences, to do presentations, to talk about their work I think is really important.
I've got a few individuals who are trying to get their first conference, their first talk accepted and to get on stage. And I've been a Call for Paper (CFP) reviewer, I know what that's like, so I can kind of coach them on that. But really it's just sharing knowledge freely and kind know. I guess that's the hacker mentality. Information wants to be free and that's kind of how we are. We share pretty much everything and there's been quite a few times when folks on my team have taught me a new thing, which is awesome. But yeah, hopefully that answers the question.
Andra Zaharia: I appreciate being so supportive and getting people to put themselves out there, especially because it is not easy. Especially when you look at the industry and you look at all of the people who've been in it for decades with such incredible research and experience and you feel like imposter syndrome not only creeps up, but it starts really shouting at the back of your head, just saying like, no, there's no point in doing this.
And just making that first step and seeing someone acknowledge and appreciate your work, that makes a huge, huge difference. And I love that you're inspiring people to do the same. And how about you? How does that kind of setting time apart to learn? How do you balance management with keeping your knowledge up to date, with making sure that your health and stays as well, as well sustained as it can be? How do you find time for all of these things? Because as we progress through all of these stages in our careers, those things really change. And sometimes they can get quite challenging.
[32:22] Willa Riggins: Yeah, I mean, I'll be honest, it is difficult to balance, especially I've got a pretty large team, but what I like to do is kind of help out folks. So we're doing a CTF team for Wicked6 this year with a few of the other ladies in the company and getting to participate with them and kind of sit with them and see how they learn and to also kind of coach and help, that's been awesome.
But also letting them watch me make mistakes because it's very humbling when you're on the command line and you're like, I can't remember what this command is. And then you work together with other folks to get there, but also finding time to volunteer to help out with CFP review for those smaller conferences and different areas of information security.
Willa Riggins: Staying involved with the community is huge, especially for me because it's a really small industry and everybody knows each other. So it's great to stay connected. And it also helps you kind of learn what's new in the space, what are people working on, what are people learning about right now. And that really helps me too.
[33:33] Willa Riggins: But also finding a hobby outside of security, I think is one of those things that's really overlooked, especially in kind of newer folks. You need something outside of your job. For me, it's going out and taking pictures of birds and things like that, but just finding something to get you away from the desk so that you can clear your mind and then come back fresh. Because I think a lot of what we do as offensive security practitioners, we spend a lot of time banging our head against the wall trying to figure out what is the next finding I'm going to get. So walking away for a little bit can be really, it's counterintuitive, but it can actually make you more productive. And I think that's really important.
Andra Zaharia: It absolutely is. I can totally attest to that. I think everyone can, especially because sometimes we learn about this particular aspect the hard way and we wish other people wouldn't have to go through the same experience and wouldn't have to put their health at risk to figure this out and to actually start practicing it.
So the least we can do is talk to others about it and also try to use our own examples, try to walk that talk and actually show people that you can still grow in a career without making it your only thing. And that's a perfectly normal and healthy way to do it. Especially, we talked about many of the aspects that go into this mindset that shapes excellence and it shapes meaningful work in this space, the things that make it a craft and not a commodity.
Andra Zaharia: What do you think are some stereotypes that you see people perhaps not question as much as they should? And how have you seen them harm people on their way to developing their careers, developing their skills, but especially developing the mindset and the motivation behind them behind it all?
Willa Riggins: Sure. I think that's really a difficult question. I think a lot of folks coming into the industry think that there's one skill they need to learn, whether it's learning how to use a command line, learning how to do web application testing, whether it's learning how to use Nmap - it can do network testing, It's not. And that's really hard, especially starting out in the industry, to figure out how do I learn all these things. There's just so much.
I think people think they need to absorb all of it, all the time, all at once. But really what kind of makes the magic is having other people who know things too. When I worked on the red team many, many years ago, I was the web application person. I did a lot of that pentest work, but we also had someone who did infrastructure and someone who did database and someone who understood OSINT. And working together, we all learned those different things, but not at the same level. That really helped me grow and broaden my skillset. But I still kept my depth in web application security.
[36:42] Willa Riggins: I think that early on in your career, understanding that you can't know everything and you're not supposed to, people make mistakes. People are going to type Nmap-h all the time. They're going to look at the help documentation, they're going to look at the Nmap pages. You're going to have to just kind of learn and grow as you do it more and more. I think that's one thing, because I see a lot of folks that are starting out and I try to kind of help where I can. But thankfully there's a lot of really great training out there to help folks get into the industry. But I don't think it fully covers the full gamut of things you need to understand, and that's kind of one piece.
I think the other two is some people view pentest as a checkbox activity. They think there's some algorithm that they can do and a process they can follow. They read the pentest execution standard and they think, oh, if I follow these steps, I can do a pentest. And that's not really how it works. I had a manager a few years ago who would draw out a diagram of what it looks like to follow the steps of the pentest execution standard. And basically it was a bunch of arrows leading back to starting all over again. And it was, you're going to do all these steps all the time, but you're never going to do them in the same order. Sometimes you're going to jump back and that really kind of shows you how much of an art pentesting can be because you have to make that decision as a human being.
Willa Riggins: Okay, maybe this application doesn't have an active directory connection, so I don't need to test for active directory. Or maybe this application doesn't have a login, maybe it's a marketing site or something. I don't need to test for login enumeration. I don't need to test for XYZ. And so kind of tailoring that methodology and understanding when to back up and then when to go deeper is also one of those things I think a lot of people struggle with, because it's not a process you can follow. There's no checklist. As much as some folks may try to implement checklists for pentests, sometimes the vulnerability is not on the list, sometimes it's not in the OWAPS Top 10. Sometimes it's something weird, completely out of left field that you have to write it up. I think that's the other thing that I see a lot of kind of newer folks struggle with early on in their career, is that everything's going to make sense. It's not.
[39:10] Andra Zaharia: Such a good metaphor for life pentesting, isn't it? It's never a straight path. There is no recipe. You barely get any instructions. A lot of things are complicated, but you're still going to figure your way through it. You're going to learn a lot of stuff. You're going to meet a bunch of great people. You're going to learn a lot about yourself, too. I feel like self-development and just maturing as a person has a fundamental influence on how we do our work and how we contribute to our community, why we do all of these things and what we get out of it. What kind of needs are met through all of these experiences and keeping that cycle going and going all over again? I feel like that's the beauty of it at the end of the day.
So thank you for reminding us of that and thank you for highlighting the underrated value of teamwork. I feel like sometimes penetration testing can attract a lot of people who like to work on their own, which is not a bad thing. But we can only learn and progress through working with others, through collaboration. We cannot work in a vacuum. It's never going to happen. We wouldn't have jobs if we have to work in a vacuum, honestly speaking, there wouldn't be things to fix if we'd all just be doing it on our own, trying to figure it all out.
Andra Zaharia: I really appreciate everything that you've shared with us so far, and I was wondering if we might touch on, let's say, things that you think will continue to be just as important going forward because people tend to get attracted a lot about the new shiny thing, the new technology, the new tool, the new methodology, the new cheat sheet.
So many things that are attention-grabbing in the space as well. But sometimes focusing on the things that don't change can turn out to be something that's really invaluable. What are some of those things that you've seen in your career so far that require attention and that are still kind of largely unsolved issues?
Willa Riggins: Sure. I think a lot of what I see in the industry boils down to having a threat model because we see a lot of new development, whether it's AI, whether it's microservices or a new cloud or whatever it happens to be. But I think a lot of folks forget to go back to basics and say, all right, where are the key components? Where are my crown jewels? What are the threats and potential vulnerabilities in this architecture? How do I look at this from a more holistic standpoint? Bring people to the table to ask, okay, what do you think could go wrong?
And I think there's a tremendous amount of value there because we can't anticipate how these new systems are going to work. We don't know how they function, what the exploits are going to be, but we do know what could go wrong. I think as humans, we can make that assertion, whether it's bias in AI or if it's buffer overflows and prompts or whatever it happens to be, we can think about that without kind of focusing on the underlying technology. We can at least think about those abuse cases.
[42:40] Willa Riggins: Because I think what's going to become more and more important in pentesting is kind of differentiating between that commodity level checkbox testing and that more human, more in-depth craftsmanship that is penetration testing. I think we're going to see a lot more focus on kind of abuse cases. How do we exploit a control? How do we get past this firewall? How do we get past this web application firewall? I think there's still going to be an emphasis on code vulnerabilities, absolutely! I don't think those are going away. I think we're going to see them in new creative ways.
But I think all of the basics are still going to be important. Whether that's the OWASP Top 10 or the CWE Top 25, whatever it happens to be, all those things are still important. We're just going to see them in new and creative ways.
But thinking about those threats instead of kind of tying it to individual technologies, I think is going to enable more people to bring more to the table. But yeah, I think everything that is old is new again every time we go through one of these cycles. But also, the threats don't change. The threats are still there. When we talk about web applications, we are still looking at access control, we're still looking at authorization and authentication. When we look at AI, we're still looking at what is the input and output that's processed. Do we trust user input? What could possibly go wrong? And I think those things are never going to change.
Andra Zaharia: That is absolutely true. If we look back at decades, even in human history, there are analogies and examples of this in real life that have continued to be problems simply because that's human nature to try to exploit things. It's all tied to human nature at the end of the day because we make technology, we shape it, we use it for the worse or for the better, and that's going to remain a constant in society for at least the foreseeable future for as far as we can peer into it.
Thank you for highlighting this. And this kind of reminds me of why it's so helpful to step outside of the office, whether at home or an actual office building, and to go to conferences, to talk to people, to have those conversations over a drink, whether it's water or something else, and to just spend time thinking about things and debating them with other people instead of just Google or another search engine or just yourself. Just getting outside of our heads in general feels increasingly important. Not that it's not always been, but increasingly so nowadays.
Andra Zaharia: So we've gone over the bunch of things that get us excited into this field and that are thought-provoking and that deserve our time, attention, and intention. But I'm really curious about what's something that you love to tinker with right now? What's something that gets you excited and gives you that feeling of, this is so new and this gives me so much energy and curiosity, I want to spend more time on it?
[46:15] Willa Riggins: Well, I've got a few of those, I think right now we're preparing for that CTF later this year. And that's been really fun because it scratches an itch that I don't get to get hands-on keyboard much anymore. So it's been really nice to get back into it and kind of see knock the rust off of some of those skills, but also to see the looks on other people's faces when they get their first exploit, when they get their first time. And that's been awesome to see.
But also just playing with hardware. I have a bunch of components behind me. I'm actually building some full-body trackers for VR, and it's really interesting to kind of get your hands on the hardware again and to tinker and solder and program and do all those different things. But also, like I said, having something outside the office, something outside of technology, just doing bird photography. Birding has been a blast, so it really kind of scratches that itch of if you've ever watched Pokemon trying to catch them all, it's like that, but they're real, and it's a lot of fun trying to get that next photograph, especially here in Chicago. It's very cold, so it's been a different challenge.
Willa Riggins: But I think that's the mix, really is looking at kind of offensive security and spending a little bit of time there to make sure I'm still plugged in, both with my team, but also with the industry, to make sure that I can still keep up, at least when we're talking about what we're doing, especially when you go to conf and things like that, you want to be able to have something to talk about, too.
But then also having something hands-on to tinker with, to build something to make, I think is really important for me at least to have something to create and to drive, and then also having something that takes you outside, I think that's huge.
Andra Zaharia: It really is. And thank you for sharing so deeply and so honestly with us all of these aspects of your work and your life. I feel like that's such a rich experience that we can all learn from and that connects something or something within us that we need. I feel like whenever we listen to a conversation, or when I listen to a conversation or podcast or have a conversation with someone else, there are things that it kind of bubble up for me and something tells me that that needs a bit more attention. That's something worth looking into, just like you mentioned earlier.
Thank you so much for this. It's been just so exciting and so motivating and inspiring as much as this word gets thrown around. But it is so, so true. It feels so true and so real. Thank you for being here and for doing everything that you do for the community. It's something that really echoes much further than it may seem.
Willa Riggins: Thank you for having me. I really appreciate it.
[49:10] Andra Zaharia: Ever wondered how deep the rabbit hole goes in the world of ethical hacking? Well, we're still falling, and we're dragging you along with us. One question at a time.
Thanks for wandering through this maze with us as we tackle the nitty gritty flipped misconceptions on their heads and maybe, just maybe, made you rethink some of the things that are important to you.
This has been the We think we know podcast by Pentest-Tools.com and before I sign off, keep this in mind.
There's always a backdoor, or at the very least, a sneaky side entrance.
See you next time.
