The Making of a Capture the Flag Competition: An Interview with David Cowen and Matt Seyer

Image of victorious person with arms raised over headAt our Magnet User Summit in May, we hosted our first ever Capture the Flag (CTF) competition. We were honored when David Cowen and Matthew Seyer, co-hosts of the Forensic Lunch, offered to organize it! Not only are they among the best known researchers in the community; their previous CTFs, as well as their Sunday Funday contests, made this bound to be a great competition—and the participants did not disagree! We sat down with Dave and Matt to talk a little bit about how they did it:

MF: What was the scenario you came up with for the CTF, and what was the genesis for that idea?

Matt: In the past a lot of our technical challenges revolved around only providing sets of extracted artifacts. When you create challenges around artifact sets, you don’t have to plan out everything at once, you can create small challenge by small challenge.

Because this challenge was going to be revolving around AXIOM, we knew we had to come up with a more elaborate challenge because we were going to be producing an image to work from as opposed to sets of artifacts. This led us down the road of scripting out the scenario that would revolve around a desktop-type environment.

Dave: We wanted to make a scenario that was more real to life than some of the technical challenges we’ve made in the past. If you are creating an image that is custom built to prove a single point, you are missing out on the investigative experience for the people playing the CTF. We wanted to create an organization that had some relation to the DFIR world, and a scenario that was more up to date with modern trends and methods of usage than just “malware ran” or “something was stolen.”

So we made “Magnetic Forensics” (since this was a Magnet event) and got them an Office365® company account, along with a GitHub account for the company. Our victim was the lead developer of the new “Magnetic Forensic” software, and we had emails between him and the other employees as we tried to simulate a real developer working on his code and pushing data to GitHub.

The usage of the system, the attack that was done, and the actions we took against the system we planned out like a movie script, so we could remember what we did and find it later.

The bad guys sent a malicious Excel® spreadsheet offering to do time conversion formulas, which no forensic developer could say no to of course!

After our bad guy (Rudi Peck, who works with us) got on the system, he handed the session off to me, where I followed the script by stealing the data via Dropbox, wiping the data with Eraser, and then dumping the users’ stored passwords in [Google] Chrome to access and delete his GitHub repository.

The idea was that this was a hostile attacker who was there not only to steal “Magnetic Forensics’” secrets, but also to sabotage them.

MF: In general, what goes into scenario-building for a CTF?

Dave: First you have to decide what kind of scenario you want to put out there, and based on our real cases there are a lot to choose from:

  • Stealing data
  • Ransomware
  • Malware infections
  • Insider trading
  • Etc…

We decided to go with stealing data and then thought up names for our companies and actors. We of course had to pick a company whose domain name we could register, and magneticforensics was available!

Once we had the scenario, we knew we wanted to make it more realistic by having the victim system run directly exposed to the internet to put more noise and real world random attacks into the mix. We were able to find a hosting provider that let us put up a Windows 10 virtual machine directly on the public internet without a firewall.

Once we had the machine up, the scenario decided, and the domain registered, we went about setting up the business services and began communicating as employees of the company, waiting for the right time for our attacker to do their thing.

Matt: As mentioned previously, the data set you will be giving the contestants plays a factor in how elaborate the scenario building process will be. Artifact-based sets require much less scenario building than an image-based set will be.

You also have to factor in how [many] resources you want to dedicate to the CTF. Before you plan a scenario, you will need to know the environment that you will have available to create the challenge.

In our case, we wanted to use cloud services to host our desktop, vs. it being on a VM within our internal network. Another question is, how personal do you want it to be? In our scenario we planned to create custom domains and utilize Office 365 just to make a “Magnetic Forensics” company so that the scenario would feel a little bit more personalized for the event. This, however, is not required and CTF scenarios certainly don’t need that level of scenario building.

MF: Did the fact that this was MUS’ first CTF affect its scenario in any way?

Dave: It did in two ways:

  1. It influenced what we named the victim company.
  2. We knew that players would have access to AXIOM, so we made sure to ask questions that could be solved with AXIOM, but also identify those that couldn’t and needed additional expertise and tools to solve.

MF: What was your favorite part of the CTF? Were there any surprises?

Matt: Watching the scores change constantly was definitely my favorite part. It was like horse races. The first place winner took first in the last seconds. That’s excitement. I made a joke with Dave that next year, since it’s in Vegas, we should have another group of people placing bets on the contestants.

Dave: I think my favorite part of the CTF was watching the scores change as the players pushed themselves to solve the questions. The top three shifted throughout the contest with first place switching hands in the last 30 seconds!

The biggest surprise to me was when our planned and tested attack plan was defeated on our test VM with a recent patch by Microsoft to Windows 10. We had to go back and change the attack plan, as Windows Defender has just gotten better and better.

MF: How did this CTF compare to others you’ve organized or taken part in?

Dave: I think compared to the challenges we’ve done in the past, this one went the smoothest. CTFd allowed us to really give flexibility to the answers to prevent frustration by the players and monitor the questions so we could see what was and wasn’t working.

Matt: This was definitely the most comprehensive challenge we have put together to date. Like I mentioned before, usually our challenges consisted of just giving out sets of artifacts to find answers from. In my experience CTFs often consist of either artifacts, images, or a hybrid containing both.

MF: What was/were the most important takeaway(s) for this CTF?

Dave: For us it was ideas on how to make this even better for our unofficial DEFCON DFIR CTF.

I think for the players it was understanding where they need to focus their studies to be able to solve all the challenges we throw at them next time.

Matt: We definitely found technologies and methods to help us create better CTFs in the future. I think another takeaway is just how much you can do within AXIOM! I think out of all the questions, there were maybe two that we couldn’t find via AXIOM. I remember I was very surprised at how fast many of the questions could be answered utilizing AXIOM.

Want to try the challenge for yourself? It’s now live and open to public access! You can download the AXIOM case file here (and if you’re not a current AXIOM user, get a trial to go with it). If you’d like to add the memory file, download that image via Dave’s blog post. The questions and scoreboard can be found here.

For more on the CTF, including questions and answers, an interview with first-place winner Jaco Swanepoel (who will talk with us more in an upcoming blog), and how AXIOM’s power made Matt worry about how fast the CTF might be resolved 😉, check out Dave and Matt’s recent Forensic Lunch:

Thanks, Dave and Matt, for your time talking to us, and again for helping to make our MUS such a great experience for all who attended! We look forward to seeing you again next year!