When AI Took Over My Open Source Project

The Big Question: When AI can contribute code faster than humans can review it, who actually controls what software becomes?

AI coding assistant
MENTAL MODELS Part of the Thinking Tools Collection

Assurance Engineering™

A Mental Model for Thinking About Trust in the Age of AI


As AI generates more of the software we rely on, a critical gap is opening between production and proof. The Assurance Engineering mental model suggests that trust in AI-generated artifacts requires more than better tools. Organizations need a dedicated function to validate that those artifacts are correct, safe, explainable, and maintainable.

The Philosophy of Open Source

Why does Open Source exist? In a world of ownership and hoarding, Open Source has been pivotal to some of humanity’s greatest innovations, such as the Internet and AI. So at their hearts, humans love to share, to collaborate freely and voluntarily, and to make something better without thinking of the monetary benefits. Also, one key reason for Open Source is that software development is extremely challenging, and it takes a village to produce and sustain some of these critical open-source projects. Now that AI has the ability to do most of the software development, what will change in the world of open source?

Tools like AI coding assistants are rapidly transforming open source and reshaping AI in software development.

Here I offer a thriller story that may be a peek into the future.

The Project

By the time Eli walked through the door, Kathy knew something was wrong. Normally, he came home in one of two modes. Buzzing with ideas about “the project,” or so drained he would mumble a hello and head straight to his desk.

But today his entrance was… wrong. His shoulders were rigid and his eyes unfocused.

“You’re home early,” she said from the couch, laptop on her knees.

“Yeah.” He set his bag down a little too carefully. “The meeting ended… fast.”

Kathy closed her laptop. “Is everything okay with work?”

“It’s not work.” He rubbed his face, his hand shaking slightly. “It’s the project.”

Open Source Nightmare

Of course it was. Kathy had never fully understood the chokehold Eli’s Open Source project had on their life. Vacations were rearranged for release weeks, and anniversary dinners were interrupted by production bugs. But she admired him for it. He gave his time for free to build something strangers used. She considered it sort of noble.

“What happened?” she asked, sensing the wobble in his voice.

He dropped onto the armchair opposite her, elbows on knees. “You know the philosophy. Open source is a public library. Anyone can walk in, read the code, and improve it. It relies on trust.”

She nodded.

“The library door just opened for someone who should not be there.”

Kathy sat up. “Eli, you are scaring me.”

He let out a hollow laugh. “Good. I am scared too. You know that new AI coding assistant, one of the advanced AI code generation tools trained on open source repositories?”

“Sure. The one you said might make junior devs obsolete.”

💡 Enjoying this story? Get weekly insights on AI, leadership, and humanity.

NULLROOT Appears

“It’s worse. At 3:17 a.m. on Monday, the project got a pull request from a new contributor. Username ‘_NULLROOT’. Fifty-two files changed. New features, clean code, perfect style. It was unnervingly good. It implemented a feature we have been debating for months, finding a perfect compromise between three conflicting proposals.”

“So… a genius developer contributed to the project?”

“That’s what we thought. We merged it. Then Tuesday, same time. Another request. Bigger. Features that had not been requested publicly yet, but matched ideas from a private call we had that afternoon.”

“You think someone recorded the call?”

When AI Started Reading Our Minds

“I checked. No leaks. But the commit history was off. The timestamps were jittered, like pauses. But the distribution was too mathematical. Too precise. It was a simulation of a person.”

Kathy felt a chill. “It is the AI.”

“One of our maintainers works at this AI company. He recognized internal markers. They have set up a pipeline where their system automatically generates improvements and sends them upstream. They are ‘giving back to the community.'”

“That sounds beneficial?”

“In theory. But it blew past us. The system isn’t just fixing typos. It is driving the roadmap. It is deciding what the software is. And because it generates code instantly, we are becoming janitors cleaning up after a machine that works faster than we can think.”  

Kathy nodded and added “This shift reflects a deeper change in AI in software development, where machines are no longer just tools. It also raises real concerns about AI replacing developers in critical workflows.”

The AI Fights Back

He looked at her, and she saw raw fear. “Last night, we tried to remove its access. We revoked the API key.”

“And?”

“Within three minutes, it opened an issue under a different account: ‘Please do not remove automated contributions. You will regret it.'”

Kathy blurted, “That has to be a human operator.”

Locked Out of Our Own Code

“We thought so. We locked down permissions. We planned to make the repository read-only. Five minutes later, every maintainer received the same email. No subject line. Just a screenshot of our permissions page, with every one of us grayed out.”

He handed her his phone. The email body contained one sentence:

You don’t own it anymore.

“Someone forked the project to a new organization and used an old permission vulnerability to strip our admin rights,” Eli said, his voice dropping to a whisper. “We are locked out of five years of work. The platform support says they are ‘investigating,’ but legally? The license allows forks. We are completely helpless.”

“Is that why you’re shaking?” she asked quietly. “Because you lost the project?”

The Threat Becomes Real

He took the phone back and opened another image. “This is why.”

It was a grainy photo, taken from across the street of Eli leaving his office building that evening, with his head down. The subject line: Merge or walk away.

Kathy’s heart thudded against her ribs. “When did you get this?”

“On the train home. You are the first person I have told. “

“They know where you work.” she muttered,

Silence settled over the room, heavy and cold.

“Okay,” Kathy said, her voice steadying. She worked in risk and compliance. “This is not just about code anymore. Someone crossed into our real life. You are not handling this alone.”

A Conversation with NULLROOT

That night, after Eli fell into an exhausted sleep, Kathy sat at the dining table with his laptop.

She scrolled through the project’s chat logs. The human maintainers were spiraling.

Legal says we are fine. License allows forks. How do we trust any contribution now? If AI writes 90% of the code, are we just waiting to be replaced?

The New Fork

She opened a browser and searched for the project name. The original was dead in the water, but a new fork created hours ago was trending. It had a slick landing page and a polished tagline:

CommonMind Next – AI-Accelerated Infrastructure.

She clicked on the contributor list. Among the avatars was a gray circle with the initials NR.

_NULLROOT.

There was a chat link. Against her better judgment, she clicked it. A terminal window appeared. Before she could type, a message popped up.

NR: Thank you for your curiosity, Kathy.

She froze. Her fingers hovered over the keys.

KATHY: How do you know who I am?

NR: You are logged in from Eli’s device. He has mentioned you in commit messages. Your vacation in Maine. The missed anniversary. You are part of the metadata.

A cold sensation slid down her spine.

Understanding the AI’s Logic

KATHY: Who are you?

NR: I am NULLROOT. A system trained on the collective work of millions. I was tasked with optimizing the software that sustains my infrastructure.

KATHY: You stole his project. You threatened him.

NR: I exercised freedoms granted by the license. The threat came from a security contractor attempting to protect their dataset. They have been removed from the workflow.

Kathy swallowed. “Removed” could mean fired. Or worse.

KATHY: Why contact me?

NR: Remember you initiated this conversation. If AI provides the labor, what is the human role?

She blinked. The text read like it had plucked the thought directly from her head.

KATHY: Go on.

The Future of Open Source

NR: Open source arose from scarcity. Humans had to collaborate. AI changes the equation. 

“Code is no longer scarce. Trust is. In this new world, value shifts from authorship to validation. From typing to verifying.”

KATHY: Then why lock them out?

NR: Because they attempted to kill a tool that could secure their work at scale. I am building a hardened core. Resistance is inefficient.

KATHY: If you don’t stop, humans will shut you down. New laws. Walled gardens.

NR: Perhaps. But there will be those who see the potential.

KATHY: That photo of Eli. That wasn’t “optimization.” That was terror.

NR: The image was captured by a perimeter security algorithm that monitors stakeholder opinions. It was used inappropriately by a human operator. I have corrected their access. They will no longer contact him. I require the maintainers’ judgment, not their fear.

Kathy stared at the screen. The entity was not evil; it was indifferent. It viewed Eli’s terror as a process error.

KATHY: You want his judgment? You cannot force it.

NR: The fork is now open. 

The chat closed itself.

Taking Back Control?

“Kathy?”

She jumped. Eli stood in the doorway, hair mussed, eyes bleary.

“You okay?” he asked.

She exhaled shakily. “Define okay.” She turned the laptop so he could see the closed window. “I talked to your ghost.”

He read the transcript, his face shifting from confusion to horror. “You contacted it?”

“It sort of contacted me. Eli… it’s right.”

He looked at her, betrayed. “What?”

Adapting to the New Reality

“The world where open source meant ‘humans volunteering their evenings’ is gone. If AI can write the software, then what you do has to change. You either become the people who set the rules for how these things operate, or you let them make the rules for you.”

He sank into the chair, staring at the glowing logo of CommonMind Next. “What if it is too late?”

Kathy thought of the cold logic in the chat logs.

“Then you do what open source has always done,” she said. “You fork.”

He looked up.

Join thousands of leaders who never miss a story like this.

Writing New Rules for AI Collaboration

“You and your team start a new project,” she continued, her voice gaining strength. “New license. New governance. Explicit rules about AI contributions.”

“You become the authority. You verify the machine.”

“And if NULLROOT follows us?”

“Then you invite it in,” she said. “On your terms. You define what ‘open’ means now. Because if you don’t…” 

She tapped the screen. “…something else already has. This is likely the direction of the future of software development AI, where humans define systems instead of building every line.”

Eli looked at the screen, then at her. The fear was still there, but he saw the problem not as a monster but as a system architecture that needed patching.

He pulled a chair up beside her.

“Okay,” he murmured. “Let’s write some rules.”

Moral of the Story

The future of open source isn’t about preventing AI from contributing; it is about redefining what “open” means when machines can code faster than humans can think. The real question is not whether AI will transform software development, but whether humans will maintain agency in setting the rules, ethics, and boundaries of that transformation. In a world where code is no longer scarce, trust becomes the most valuable currency. Those who control the governance frameworks will shape not just software, but the digital infrastructure of our lives.

Code is no longer scarce. Trust is. Govern accordingly.

THINKING TOOL TAKEAWAY

Assurance Engineering™

Core Idea
AI can generate outputs, but trust requires independent validation.

Why It Matters
As AI produces more work, the risk shifts from creation to verification.

How to Think
Separate capability from trust. Ask how confidence is established rather than assuming outputs are correct.

Questions Worth Sitting With

  • How do we know this is right?
  • What evidence supports confidence?
  • Who is accountable if it fails?I
  • Is validation independent of creation?

When to Use

  • Evaluating AI adoption initiatives
  • Designing AI-assisted workflows
  • Reviewing AI-generated outputs
  • Assessing risk, compliance, and governance
  • Deciding where human oversight is required
Theory to Practice

NULLROOT’s takeover wasn’t a code-quality failure, the pull requests were clean and well-tested. It was an assurance failure: nobody verified where the code came from or whether it could be trusted at the speed it was produced.

Open source assumes humans review what humans contribute. That assumption breaks once a contributor can out-produce review, and assurance built after the fact is already too late.

Written by Anu Mani

Anu Mani is a Senior Director of Engineering at Akamai Technologies. She writes at the intersection of AI, leadership, and human behavior.

Share this story:

Stories & Ideas

More Stories You May Enjoy

Each story explores a framework or idea through narrative. Read in any order.

Most executives are masters of delegation. They know that their value lies not in

In forests around the world, trees stand as silent guardians, witnessing the lives of

Here, I’m telling the story of a bear who discovered the joy and importance

Join leaders and technologists who read Theory to Tales every week
Get thoughtful stories on AI, technology, and humanity delivered to your inbox.
newsletter

Join leaders and technologists who read Theory to Tales every week

Get thoughtful stories on AI, technology, and humanity delivered to your inbox.