Gaby Friedlander: Good morning, good afternoon, good evening, depending where you're joining us from. Welcome everyone! Today, we're diving into some exciting ways to keep engagement high in your phishing simulations and training sessions. Joining us, we have James, who'll introduce himself in a moment. James came up with a fantastic idea that we recently tested, and I'm thrilled to share the results and their value with all of you.
We also have Ayelet here, our Security Awareness Manager, who works with our customers doing our Managed Security Awareness services to our customers using Wizer Managed. Ayelet guides them through these activities, offering her expertise and support.
James Linton: Thanks for having me on. I'm James Linton, a social engineer who's kind of semi-retired now.
I gained a bit of notoriety for tricking the White House and banks and stuff like that, which is mildly embarrassing now. But it sparked my interest in email and the psychology behind why I was able to convince people I was someone else.
Was it because I'm a psychological genius? Turns out, not really. It's more about people being so used to seeing the same display name over and over again that they automatically trust it, even the thousandth time. So, I think that has a lot more to do with it. That's why I love exploring different aspects of security, awareness, design, and ideas through experimentation.
The Concept Behind BattlePhish
James: You were kind enough to suggest trying out a pilot of BattlePhish, just to see how it goes. The concept kind of popped into my head one day, thinking about phishing simulations. One issue I found with them is the lack of transparency. It can feel like a covert red team exercise if you don't understand what's happening behind the scenes. But I don't think it's as malicious as some employees might think. There's actually good intention behind it. So, I thought it would be interesting to let people see behind the curtain. That's why I came up with the idea of splitting into teams and picking an email.
The idea behind picking an email is to get the individual considering, "Will this trick me?" instead of "Will this trick Gaby?" Each person tries to find the hardest email to send the other team.
I thought it was an interesting twist that people would now study and analyze three different emails to figure out which one would be most likely to trick them. It seemed like a cool concept to explore, so we decided to give it a try.
In working with Ayelet we decided not to worry about perfection. We’ve got to get things wrong before we can even get something right. So I kind of pulled away all the parts that were extraneous to it and I thought, what's the minimum kind of MVP that we can get rolling.
Two Sides of the Same Phishing Coin
Gaby Friedlander: What I really liked about this whole idea - and I'm going to repeat some of the things that you said - is that traditional phishing is perceived as an attack on employees.
A lot of employees don't like it. And this idea gamified phishing which is actually pretty cool. This turns the table. This makes phishing a game where I need to kill the phish instead of “I am being attacked”.
People know that they're participating, whether they like it or not, but they are aware of it. And there is a sense of accomplishment at the end and a winning team. What I also liked about this is that because you split it into two teams you get to sit on both sides of the desk.
You're also an attacker. You could either pick a template out of many or craft your own. We put that aside for now because crafting just added a little bit more complexity. However, when the teams started engaging, some of them responded with custom ideas. So the discussion itself was actually really interesting.
You're basically an attacker and at the same time also the victim because the way it works is that there are two teams. Each one of them picks which template they think the other team will fall for. And then each team sends the other team that template.
Then we see who clicked, who reported, and calculate a score out of that. So that's like the big picture of this game. I'll let you now go into the details more of how did we set it up?
James Linton: I'll pass over to Ayelet at this point, just because obviously the first question to have was we need to be able to send out a phishing simulation, which was easy enough because we had the departments, but then we also had to send out a way of voting to everybody outside of that.
Selecting The Teams
Ayelet Penrod: There's so many ways you can break up the teams, but because this was kind of our first gamified group effort, we thought to keep people with their departments to lend to a sense of camaraderie a little bit but to also see the different types of demographics to target. So it wasn't just a random mess of people, but more kind of role-based. We decided to break it up between our technical side - the devs, the engineers - versus the more people-facing roles of HR, Sales, Customer Service, etc.
We decided just to keep the voting really simple with a Google form and an announcement in Slack first to let people know an email was coming that required their input and what email subject line to look for so they don't think that's a phish.
Gaby Friedlander: So the Google form for each team included multiple email templates to choose from and they were to vote on the template they wanted to use to attack the other team.
The Weapon, er, Phish of Choice
Ayelet Penrod: You're right. We tried to mix it up. I'll let James talk more on that, but we tried to give a different approach for each template to kind of see what people would select.
Gaby Friedlander: Which is also actually pretty interesting. The template that were chosen were on the technical people team, they chose a HubSpot form, right?
James Linton: Yeah.
Gaby Friedlander: They were like, okay, we think the non-technical people that use a CRM and stuff like that will probably fall for something like HubSpot. So they tried to make it relevant, which is also cool because they didn't just randomly pick something. They were putting some thought into this and they knew the demographic and they knew what they might more likely fall for.
James Linton: In the group of phishing template options we included Netflix. And although it seems a bit of an odd one, I thought of putting that in as it shouldn't really turn up in a work inbox, should it? If you had a shared inbox or something then perhaps yet, but as it turned out it got zero votes. They didn't think that was that's all plausible, which is interesting in itself, I think.
So Wizer’s obviously got a very clean bill of health in terms of people mixing their work and personal emails together.
Ayelet Penrod: They had Netflix, the HubSpot, and Loom. Loom and Hubspot were the two tools that were voted on the most, but yeah, the HubSpot had the overall, widest margin of votes from the Dev team (Team A) to send to Team B.
Gaby Friedlander: So what about the non-technical? What did they do?
James Linton: HR, operations, customer success, and marketing. They chose a classic really. It's caught out many people over the years - a Google workspace, shared document file template. I mean, ubiquitous because if day in, day out to collaborate, you're almost paid to open any email, I guess, to a certain extent, but things like that can be super tricky.
And everyone on the non-technical team sort of went for that particular option. There was an IT message there that also got a couple of votes. The WeTransfer template was also thrown in there about a new promo video because I know that Wizer is king of the promotional PSA videos, but nobody voted for that.
They all just dived straight for that big hitter of the Google Workspace template.
Need new Phishing Simulation template ideas? Check out our featured Phishing Email monthly series.
Gaby Friedlander: They didn't expect to get it over the weekend or during a workday. What I also like is that both teams picked something relevant. They understood the perspective of an attacker in that you’d want to choose something that will look the most legit for that clicking.
James Linton: This is the really interesting thing for me. You never really get somebody's opinion about things like this. They actually look at an email and have to think, would this be likely to trick me?
The Timing
Gaby Friedlander: Yeah, go ahead Ayelet, explain what happened next. When did you launch it? Did you do it immediately after? Did you wait a few days?
Ayelet Penrod: So we had a little lag in voting. I actually had to wait till over the weekend and tried one more push to get a little bit more engagement with the voting and after a few more came in we took everything and just launched it. We put it into our phishing simulation and launched it to both teams, and then just kind of let it run from there.
Gaby Friedlander: How long did we run it?
Ayelet Penrod: It ran just over 48 hours. One thing I wanted to say though, when we sent out the forms for voting, I had one of the team members, immediately write back, ‘hey, I have a suggestion. We should try a phish like this’. And the phish she proposed was actually built based on the phishing competition.
She wanted to make the subject of the email about the competition itself - the BattlePhish. She was going for the really hard hitting ones. Unfortunately just because of the timeframe we had, we decided to focus right now on a more standard, templated version. I loved the engagement though.
She happens to be one of our top phishers in the company, by the way. She's caught more than one suspicious phishes for our customers or vendors and notified them. So she's really on top of it.
She's definitely a person I would want to reach out to later for a champions program.
So we had HubSpot facing off against Google Workspace - the vote was decided.
Scoring The Game
Gaby Friedlander: Before we talk about the results, let's talk about the points system. How do we decide who is the winner?
Ayelet Penrod: James was the one who came up with the actual calculation of it but we wanted to emphasize not just the clicking but also the reporting.
As part of the Wizer philosophy reporting is really the main KPI. So we wanted to reward people for that as well. I'll let James explain more of the math part.
James Linton: Yeah, in the beginning I was only looking at it as more of a hacking exercise.
It didn't even occur to me that you could also include the reporting as well to get points so it was such an amazing idea.
That’s what I like about this overall concept is that anyone can kind of take it and add their own ingredients to it and customize it for themselves.
Gaby Friedlander: So the points system is one point for one action?
James Linton: Obviously we would have a percentage of people who reported it and a percentage of people who clicked on it. So essentially there was a maximum score of 200 points possible, if everybody reported it and all the other team clicked on it.
Then that percentage added onto your overall total. It was never going to be anywhere near 200 points, but that was the sort of thing where, if some people clicked on your team that 'loss' could also be mitigated by other team members who reported it.
Gaby Friedlander: So Team B gets the phishing email from Team A. Five people clicked on it. Does Team A get five points because they tricked five people from team B to click?
James Linton: Yeah, they would register as a percentage of people that clicked out of the campaign which would award points to Team A. So if it was 10 of the opposing team that clicked they would have 10 to add to their percentage of reporting rate.
Gaby Friedlander: Every company can come up with whatever they want. What we suggest is calculating both the Clicks and the Reports and come up with a number.
James Linton: One variation of scoring could be that if somebody clicks on a phish AND reports it, that could be worth double points. It potentially takes a bit more boldness to report it after a click.
There's loads of little tweaks you could make on how the points are done. To be fair, we got the stuff in the dashboard and then we figured out how somebody was going to win it. While it was sort of obvious because we could see who was doing the best we wanted to sort of have a really simple way of giving it a score to what had happened.
And I think it reflected the kind of overall trends. Another interesting thing was that Team A - the technical group - only had 43% actually open the email, whereas a full 70% of the people in sales, HR and operations opened their email.
Now we don't know if they were opening it in order to report it because they knew it was fake and maybe the technical people were kind of suspicious and decided not to even open it. Or was it because they were using different channels for communication mainly and naturally check their email less frequently. Again, these are questions that would be interesting to kind of follow up as a group.
The Results
Gaby Friedlander: Let's start with which team won and then we'll break it down.
James Linton: The winning team was Team B - the non-technical team - Sales, HR, Operations, Customer Success, and Marketing. They really stormed away with a kind of rounded performance.
They had a high Report Rate of 22% which was pretty good. Plus, they managed to catch out two individuals on the opposing team and got them to click on their email - which again was the Google Workspace form. It could have been the form that tipped the balance but again the other team if they'd have been quicker on the reporting, they could have mitigated the results.
Two people clicked from Team A and this was quite interesting. Obviously there's absolutely no blame meant to be associated with this game, but it was interesting to note they were both new hires at Wizer. So it could be they were simply just super eager to reply to everyone that's getting in contact with them and getting used to all the systems and stuff.
Everyone else is kind of weathered and toughened to kind of phish as they would be working on building a platform.
Ayelet Penrod: James, the top three on Team A who did report are our veterans, right? They’re the long-time employees and maybe that they worked on the phishing simulator itself. So it's hard to know and that's where the conversations would be good, that I would like to have a follow up. My question would be if they just happened to recognize as a work template or they genuinely just saw it as a phish and flagged it.
James: I think being able to open up a dialogue between everybody about these kind of things is something that wouldn't kind of come up in another way in that kind of exchange of information especially around a phishing simulation.
You learn a lot from people's perceptions of stuff as well, or hopefully you will. Whereas normally a phishing campaign, you fire it off, see if somebody clicks on that and you don't really get much value from it, but I don't think it really sparks a bit of debate.
So doing this exercise maybe once every six months, once a quarter is good. And I think one of the other things that drew me to it originally as a concept was that if you've had a go at doing it yourself, then maybe you're not going to see it as this bad thing afterwards and there'll be more trust there with the process a bit more transparent. And I think that's, that hasn't really been experimented with enough in phishing simulations so far.
Ayelet Penrod: I was just going to say also, just like we have for our dev training where we teach developers how to think like a hacker - go on the offense to learn defense kind of thing - that that's kind of what this game does. It allows the individual to think like a hacker a little bit, reevaluating what would really trip up the other team? what do I need to think about? Their work environment and their different tools and stuff.
Giving that opportunity so that hopefully, like James was saying, that will also help them to be more on the defense, just in every day.
To Summarize
Gaby Friedlander: To summarize. I think first of all, the fact that it's a game and people understand that they're not being attacked - that they're just part of something. It’s also opening their eyes more because they don't know which template, maybe it's a real phish they reported, which is great.
So they have their eyes on their emails. They're more aware. And I think it opens engagement. So what I would probably do next time I would open a Slack channel. For each team, just to let them talk a little bit more, maybe if we give more time, they can come up with their own template as well.
So you're part of it. I really liked the fact that you get to play both roles, the attacker and the victim. The fact that you get to practice different things, you understand the importance because you're going to get points for reporting. So it really emphasizes how much we care about reporting, which is just as much or even sometimes more than the fact that you clicked.
It's not a pass or fail exercise on the click. You can mitigate what you've done. And I think the concept that you can mitigate what you just did is a very powerful message that you don't need to hide. There is really no need to think, ‘Oh gosh, what did I do? I will pretend that nothing happened.’
Those things are things that are very unique that we don't usually get to celebrate during a regular phishing simulation. Usually there's not a lot of celebration, which sucks because it's a drill. It's something that we can actually celebrate success and there are successes reporting.
So again, I think this is really unique. I really like it. We'll do it more. And the great thing about this is that anyone can do it. We have our own phishing simulation, but the concept that we just talked about can be done with any phishing simulation. It doesn't matter.
And even if you don't have a phishing simulation, you have a marketing tool. You can use that to send an email to two groups.
We have two more minutes, so I'll let each one of you give a sort of a closing statement. And first of all, James, thank you very much. And Ayelet, and it took some effort and time, and I think this was a great exercise.
James Linton: There's loads of ways that somebody could run with this. And like I said, it costs kind of nothing but you have to be willing to step away from, that lovely, perfect clicky graph and show people some emails and let them study them and decide which you're going to use to try to trick them.
Ayelet Penrod: We definitely have plans for version two next time.
Gaby Friedlander: Okay, guys. Thank you very much for attending. Hopefully this was useful, try it out in your own companies and let us know how it went. Until next time.
