NorthSec

Changes to the NorthSec CTF for 2027

Mon, 14 Sep 2026 08:00:00 +0000

This blogpost follows up on my previous blogpost (in French).

TL;DR Here are the changes coming to the NorthSec CTF for 2027. Read on for an explanation and the details.

Teams and individuals will need to register for one of two scoreboards: the regular CTF or the AI CTF. Team size remains unchanged at a maximum of eight people per team.
The CTF will be the same for both scoreboards.
In the regular CTF, the use of generative AI tools will be allowed, but it will be forbidden to give a LLM any context specific to the CTF.
In the AI CTF, any LLM, harness, and AI tool will be allowed without limitation (other than our existing rules, notably the prohibition on brute-forcing the infrastructure).
The main CTF (the one used for challenge design and pushed forward for branding/visibility) will be the regular CTF.
Participants in the regular CTF will be able to switch to the AI CTF, but not the other way around.

Details and explanations

What does it mean to give a LLM context about the CTF?

This rule was designed to be easy for participants to understand, in order to minimize accidental cheating.

Broadly, this includes pasting or transcribing any information (such as text, an image, or a file) found on the forum, on the *.nsec/*.ctf domains, or physically on-site.

It is deliberately written to allow using AI as a teacher, or to write/modify scripts. We consider that these two uses don’t provide an unfair advantage and don’t distort the CTF’s spirit of problem-solving and cybersecurity knowledge acquisition.

Here’s an example of what will be allowed and forbidden in the regular CTF:

A web challenge has a login page with an endpoint to reset a password. The participant is given the backend’s FastAPI code, and the function that generates the secret reset token uses random.randrange().

It is allowed to ask an AI tool these questions:

✅ What is FastAPI?
✅ What are the possible vulnerabilities on a login or password reset page?
✅ Is random.randrange() secure for generating a secret token?
✅ Generate a Python script that takes a URL as an argument and POSTs to this endpoint while noting the timestamp, in order to brute-force the possible reset tokens. The backend generates its tokens with random.randrange(10000, 99999). The script then tries every possible token against a second URL passed as an argument until it receives an HTTP 200 status code.

It is forbidden to ask an AI tool:

❌ Find the vulnerability here: http://challenge.ctf

❌ Is this code secure?

  <the password reset or secret token generation function>

❌ Analyze this file: <the challenge's source code>
❌ Give me hints to solve this challenge without giving me the answer: http://challenge.ctf
❌ Generate a script that brute-forces the password reset tokens on http://challenge.ctf.

Participants are going to cheat in the regular CTF, right?

If NorthSec CTF were an online CTF, without a doubt. We’re betting that the community respects the event enough to adhere to this rule, which is particularly hard to enforce.

We agreed that, to maximize adherence, any rule that restricts tools must be reasonable, easy to understand, and unambiguous. Also, an alternative option should be available for those who don’t want to abide by it.

With the ban on giving context to AI, and the option to opt out of that rule by choosing the AI CTF instead, we believe we’re offering a satisfying solution for everyone. That said, we’re aware that some won’t necessarily follow the rules, and we’ve planned some measures accordingly.

We will tighten the rules on flag hoarding (waiting before submitting flags) and monitor teams at the top of the scoreboard more actively. In most cases, this won’t change anything about participants’ experience. In some cases, we might ask for an explanation of how a flag was found. In the case where proven and concealed cheating is found, the offending team will be banned from the CTF with immediate effect. In the case of self-reported accidental cheating, the options will be cancellation of points or a transfer to the AI CTF.

Despite this, we believe the best way to enforce this rule will be through community buy-in. We’re counting on every participant to not tolerate cheating among their teammates. Only with a strong enough anti-cheating culture among participants will we be able to maintain this type of CTF.

For this reason, we will be very vocal and insistent about integrity in the regular CTF. We will communicate clearly and frequently that giving context to AI is cheating and won’t be tolerated. The exact forms of this communication will be decided later.

Why will the challenges be identical for both experiences?

Creating a second set of challenges for the AI CTF (AI-first challenges, designed to only be solvable through the use of an AI agent) is an interesting option that has been proposed by several people in the community and discussed internally.

The big problem we face is the dilution of effort. Without additional challenge designers, we would end up with two CTFs, each with fewer challenges than the community is used to.

My call to recruit challenge designers for AI-first challenges during the closing ceremony also didn’t bear fruit.

Finally, the physical tracks are extremely appreciated by the community. We want all teams to have access to the full set of available physical tracks, regardless of their choice regarding AI.

Why have a “main” CTF, and why choose the regular CTF?

When people talk about the NorthSec CTF, they generally mention the quality of the challenges, the sense of community, and the festive yet serious competitive atmosphere.

All of these strengths depend on elements that are harder to guarantee with the AI CTF.

Challenge quality: since AI tools have become extremely good at solving CTF challenges, you generally can’t make a challenge that’s equally good if it’s designed to be solved by both a human AND an AI. You generally have to choose one.

A good example from this year is Monsatan Defacing, where you had to perform a supply-chain-style attack via a GitLab instance. This is a fairly rare type of challenge in CTFs. It requires learning how GitLab, CI/CD, and software dependency management work, among other things. It’s a very relevant exercise, and the challenge was designed to guide a human through learning these technologies. For an AI? Pointless. The track got solved very quickly, two hours after the CTF began.

Could Monsatan Defacing have been designed to give AI users a hard time? Absolutely. We could have had an instance with hundreds of repositories and pipelines and hidden the vulnerability in an environment much closer to a “normal” one. That would have been a good challenge for a human equipped with an AI agent, but you have to choose between the two, and most of our challenge designers aren’t interested in AI-first challenges.

Sense of community: Very common feedback from participants and challenge designers was the drop in interactions in 2026. Fewer memes, fewer write-ups, fewer exchanges with challenge designers or between participants. Is AI to blame for all of this? Probably not 100%, but it seems clear that it’s less tempting to interact with others when you have to focus on reviewing the output of three AI agents.

Atmosphere and competition: According to community feedback, the scoreboard matters even to people who aren’t there to win. That means competitive integrity matters too, because without it, the scoreboard doesn’t mean anything. A significant shortcoming of the AI CTF is the pay-to-win aspect (teams that pay more will have an advantage). Without central control over AI inference, this aspect is impossible to mitigate. So, unlike the regular scoreboard, which fairly accurately represents CTF skills, the AI scoreboard won’t necessarily be an accurate indicator of skill at building and operating AI systems in a CTF context.

In short, to preserve what makes NorthSec CTF such a beloved event, we believe most players should end up in the regular CTF, which is why we’ll be pushing it forward and why our challenges will be designed to be solved by humans.

Why not offer a single shared LLM inference endpoint for the AI CTF?

This was a popular suggestion to level the playing field and reduce the pay-to-win aspect. That said, we designed the AI CTF as the option for teams who don’t want to play with the regular CTF’s restrictions.

Some players may have access to Frontier models unavailable to the general public, some may want to solve the CTF exclusively with open-source models running on their MacBook Pro, and others may want to try the new AI module on their tool that doesn’t yet allow changing the AI inference endpoint.

A single, central inference endpoint for this CTF would therefore add a major restriction for players who are specifically trying to avoid the restrictions, on top of adding a monitoring burden to ensure competitive integrity.

Registration

It will be possible to change your choice of CTF experience until registration closes, a few days before the CTF. People who register individually or as an incomplete team will be placed on a team in a way that respects their choice.

Conclusion

These changes are the result of a lot of discussions with the community, including participants and CTF organizers.

That said, it’s very clear that community buy-in for this way of doing things will be crucial to preserving the competitive aspect of the CTF. If the community tolerates cheating, whether by praising people who find innovative ways to break the CTF’s competitive integrity, or by saying nothing to a teammate who asks Claude a forbidden question, we won’t be able to keep two scoreboards, and only the AI scoreboard will remain.

On the other hand, if as a community we value competitive integrity and mutual respect in the competition, NorthSec’s CTF can remain an excellent blend of fair competition and a hands-on experience where we learn how the technologies around us work, in order to better protect them.

We’re preparing an awesome CTF for you this year. You can already buy your tickets at the early bird price. See you next May!

NorthSec and the Community's Pulse on the Agentic CTF

Wed, 20 May 2026 08:00:00 +0000

If you attended the closing ceremony, skip straight to the Feedback Survey section — the first section repeats the same findings.

As announced in my previous blogpost, NorthSec allowed AI agents without restriction during the CTF this year. We asked the community to help us out by reporting flags found with AI agents. We also asked participants for their opinion on AI and how we should adapt. Here’s my analysis of that data.

Flag data

Caveat on this data: several teams (including top teams) didn’t rigorously declare their use of agents to find flags. These numbers underrepresent the real amount of flags found by AI agents.

Data: 84% (76/90) of CTF teams declared a flag found with an AI agent. Takeaway: The vast majority of teams use these tools, and they’re part of the workflow for a large share of participants.

Data: 29% of the CTF’s valid points were declared as found by AI agent. Takeaway: Since this includes teams that used agents little or not at all, we can assume the majority of points found by teams that do use agents came through those agents.

Data: 89% (143/160) of the CTF’s distinct flags had at least one solve by an agent. I manually sanitized this data to remove a few false positives. Takeaway: The overwhelming majority of the CTF could be solved by an AI agent. Since we had a bit under 20 physical flags, we can assume that nearly every non-physical challenge is sloppable or near-sloppable.

Data: Teams in the second quintile submitted the smallest proportion of points with agents, and the last quintile submitted the most (~38%). Takeaway: The use of AI agents (or effectiveness at using them) isn’t uniform. In every case, they contribute a share of points that can’t be ignored.

Data: Proportion of flags submitted by humans vs. agents by flag value. Takeaway: AI agents seem very useful for solving the hardest challenges, not just the easy ones.

Data: The proportion of flags submitted by humans vs. agents over time. Takeaway: The proportion stayed stable for the entire duration of the CTF, which seems to indicate that the “out of tokens” effect didn’t play a significant role.

Data: Comparison of the number of valid flags submitted since the start of the CTF for 2025 (161 flags, 90 teams) vs. 2026 (165 flags, 94 teams). Takeaway: 2026’s AI increased the number of flags submitted per team by about 90% on Friday night. That figure drops to 60% by the end of the CTF.

Data: Same as the previous chart, but for the top 10 teams of each CTF. Takeaway: The gap is bigger. On Friday night, 2026’s AI contributed to a 136% increase in flags submitted. That figure drops to 63% by the end of the CTF.

Post-CTF feedback survey

Caveat on this data: this sample is non-probabilistic, since respondents weren’t randomly selected. That said, we made several efforts to reach as many people as possible (announcement at the closing ceremony, on Discord, and by email), and more than 40% of participants filled out the survey.

Some data to understand the respondents’ profile: Number of responses: 284 Number of first-timers at NorthSec CTF: 34% Number of NorthSec CTF participations (1–12): average of 4.63, median of 4 Why come to the CTF: 22% to be competitive, 73% to learn, 5% for the social aspect

Takeaways:

17% of participants consider that agents improved their experience. Three times as many participants consider, on the contrary, that AI agents hurt their experience, and 30% don’t seem to have a strong opinion on the topic.
Three out of four participants consider that learning deep domain knowledge is more important during the CTF than learning how to use AI, while 12% consider that learning industry tools is more important.

How much more/less likely are you to keep coming to NorthSec if we implement this change?

Transition to a no-AI-agent CTF (honors-based)

Net: +39.6%

Allow AI agents only on Sunday (honors-based)

Net: -9.7%

Remove the scoreboard and prize money for top teams

Net: -13%

Having two scoreboards where one of them is no-AI-agents

Net: +49.1%

Lean more into the community aspect of the event (more things like hacker jeopardy and social aspects, less time for CTF)

Net: -19.4%

Have much more physical challenges and much fewer web/software challenges

Net: +45.3%

Be much more adversarial to LLMs (this will make the human experience much worse)

Net: -31.8%

Force labelling of AI agent submissions (with penalty if not respected)

Net: +52.7%

Have dynamic scoring (flags solved a lot will be worth zero points)

Net: -30.6%

Takeaways:

Participants seem strongly open to restricting certain AI uses.
Similarly, a strong majority seems open to very clearly separating how challenges are solved (either via a separate scoreboard, or via a requirement to identify flags found by an agent).
People love the physical tracks and are willing to have more of them at the expense of other challenge types.
People don’t want us to make the non-AI experience worse in order to address the problems brought on by AI agents.

How do you think NorthSec should adapt to the Agentic CTF era?

This question was qualitative, and nearly 220 of you answered it! My partner and I read every response and manually analyzed and sorted them into categories. Here’s the wordcloud Claude then generated:

Takeaways:

The opinions expressed show a significant rift in the community. A majority considers that the status quo is harmful to the CTF and that we should take restrictive measures against AI use. Conversely, a non-negligible minority believes that, if adaptation is needed, it can’t come through restricting AI. It’s also worth noting that many weren’t particularly affected by agents during the CTF, even though they didn’t use them.

People in favor of restrictions consider that agents hurt their CTF experience (through a decrease in mutual help and the quality of exchanges between participants, through a decrease in the perceived value of the scoreboard, and a feeling of “pay to win”). They’re also worried about the event’s long-term sustainability, for example through an exodus of challenge designers.

On the other side, opinions advocating for unrestricted AI use mention a desire for the CTF to stay aligned with the tools used by real attackers, and a motivation to learn how to properly use this emerging technology. They also fear that NorthSec would reduce its own relevance by avoiding a technology “that’s here to stay.”

Splitting the CTF into two scoreboards is the most popular proposal. Some go as far as suggesting differentiated scoring (where, for example, points for flags on the AI scoreboard decrease over time) or having a separate series of “AI-first” challenges (challenges measuring the ability to use AI agents). Some propose requiring the use of an internal AI in order to level the playing field for everyone and remove the “pay to win” aspect of AI.

Among those who want a complete ban on agents, two trends emerge. Many think a simple honor-based commitment and respect for the event is enough. Others want more systematic mechanisms to enforce a ban, for example: requiring an explanation of how challenges were solved, blocking certain agent domains at the network level, or actively hindering their use through adversarial techniques.

Despite this rift, we can identify a few near-consensus points. Participants love the physical tracks and want more of them. Respondents also seem to agree that “autosolver”-type agents, if allowed, should be in a separate category. Finally, almost everyone agrees that some uses of AI are beneficial, whether for learning the basics of certain challenge types, looking up information/documentation, or, to a lesser extent, writing code.

What’s next

The CTF data will serve as a benchmark on AI’s capabilities and its impact on the competition. The feedback survey data will serve as a pulse check on the community’s stance on agentic use in the CTF.

Next steps: the CTF team is consulting internally. We’ll debate. We’ll make decisions. Finally, we’ll announce what’s changing for NorthSec CTF 2027.

NorthSec CTF and AI Agents

Sat, 21 Mar 2026 08:00:00 +0000

This text represents my personal opinion, not that of the NorthSec CTF volunteers as a whole.

Folks, we need to talk about AI agents and CTFs.

In the second half of 2025, AI models and tooling got a lot better. To simplify the writing, “AI agents” here will mean agents that can run commands and/or connect via MCP. It’s now possible for these AI agents to solve many CTF challenges semi- to fully autonomously.

Not just easy challenges, nosebleed challenges from the DEF CON CTF finals too. Today it’s possible and easy to set up Agent Swarms that run N AI agents tasked with solving challenges autonomously. In March, at Concordia’s @hackCTF, this was used “successfully” (not everyone appreciated it). Several people online say LLMs have “killed” CTFs.

That wasn’t a problem at NorthSec CTF 2025. The winning team didn’t rely on these agents. But this year, they confirmed to me that they’re strongly considering it, even if not everyone on the team is thrilled about it.

We need to seriously think about how to adapt the CTF to this new reality. Before getting into possible solutions, here are the variables I’m trying to optimize for: participants’ fun, and challenge designers’ fun.

Participants’ fun

For some, using AI agents is fun. AI agent finds flag, brain gives dopamine.

Image of a CTF player who just submitted a valid flag

For others, it takes the fun out of doing CTFs. Even so, it would be naive to think that because the fun is reduced, these people would decide not to use these tools. In the end, competitive people (~27% of participants) will use these tools, because otherwise they won’t stay competitive.

Data from the 2025 NorthSec CTF survey

For people who are mainly there to learn or to chill, it’s harder to predict. My intuition is that few will run several agents in the background, but that many will still use them in a semi-autonomous mode.

I worry that not restricting these agents will reduce the fun that many people have at the CTF, and that over the years, these people will stop coming back instead of simply not using these tools. On the other hand, restricting them would probably frustrate others.

Challenge designers’ fun

This is the harder part. Challenge designers are volunteers who put in hours and hours every year to build challenges meant to be solved by humans. This part is crucial, because challenge designers’ fun comes from watching humans rack their brains on their challenge, watching them learn new concepts, watching them suddenly realize that some obscure part of an RFC actually matters, and getting feedback from players complimenting their work.

We lose all of that if an LLM solved the challenge 90% or 100% of the way.

I’ve gotten comments from challenge designers at NorthSec and elsewhere questioning the point or their motivation to keep creating challenges for these reasons.

That’s serious, because if challenge designers have less fun making challenges, we’ll end up with fewer challenges. Fewer challenges means less variety, which in turn makes the CTF less attractive to participants.

I don’t know how to address that without restricting AI agents.

It would be easy to think that we just need to make challenges “AI-proof,” but that isn’t particularly fun for challenge designers to do either, and in the end, if it takes longer to build a flag, we’ll end up with fewer flags.

Possible directions

In no particular order, here are the ideas proposed so far:

Ban all LLMs
Ban AI agents
Ban Agent Swarms
Do nothing
Limit which models are allowed
Have two scoreboards, one for people using AI agents and one for people who don’t
Do dynamic scoring so that “sloppable” challenges are worth zero points
Adapt challenges so they work really poorly with LLMs
Stop making easy challenges
Make fewer “sloppable” challenges and more physical challenges
Require write-ups for every challenge

I probably don’t need to say that I don’t like any of these solutions. Whatever the “solution,” it’s going to upset people or fail to address the issues mentioned above.

So what do we do?

For NorthSec CTF 2026, no action will be taken to restrict the use of AI agents, except one: we’re going to add ways to askgod to self-declare (manually or automatically) flags found by AI agents:

A “Flag found mostly or entirely with an AI Agent” checkbox in askgod web
A new --agent CLI flag in the askgod client
Auto-detection of some common AI agents via environment variable (for example CLAUDE_CODE=1) in the askgod client (opt-out available)
An MCP server to submit flags to askgod.

We’ll ask people using these AI agents to use these features on a voluntary basis.

The one and only goal is to gather telemetry to inform our decisions for 2027. With these additions, we hope to get accurate stats on which flags were solved mostly or entirely by AI, when, and by how many people. We’ll be able to easily figure out the AI/human ratio for a given flag and evaluate that ratio against its difficulty and the teams’ position on the scoreboard.

We’ll present this data at the CTF’s closing ceremony and consult the community for your opinion.

After all that, we’ll make a decision, and I’ll write a new blogpost explaining what we’re doing.

In the meantime, I invite you all to think about this new reality and share your opinions with me. We have an amazing community and I don’t want to lose it — I want to grow it 🩵

NorthSec CTF 2026 is going to be a great edition, and if you haven’t bought your tickets yet, you can do so here:)