Article
When Your AI Agent Joins the Team
From learning to access control
In my previous article, I described building an AI agent that learns between sessions. An agent with structured memory and a self-correction loop. A system for accumulating experience across runs.
The moment other people needed access to it, the problem changed completely. It was no longer about whether the agent could learn. It was about who gets to teach it.
What happens when your team gets access to an AI agent
Most articles about AI agents focus on what the agent can do. Very few talk about what happens when someone other than the builder starts using it.
I built a Slack bot. The idea was simple: give the team a way to interact with the agent directly, ask questions, request analyses, get reports. No terminal. No code. Just Slack.
It worked immediately. And that is when the real problem appeared.
Free chat with an AI agent is a powerful interface. It is also a risk. If anyone on the team can write anything to the agent, anyone can accidentally overwrite its memory or change its behaviour. They can trigger actions that were not intended. The agent does not judge authority. It processes input.
The question was not whether the team should have access. The question was: what kind of access?
Role-based access control for AI agents
I ended up with a role system. Three levels: admin, analyst, viewer.
The viewer can read reports and see what the agent produces. Nothing more. No commands, no chat, no influence on behaviour.
The analyst can do more. They can ask questions. They can run predefined commands. And, this is the important part, they can write to the agent's memory. But only through an explicit command, not through free conversation. If an analyst types a memory instruction in the right format, the agent saves it. If they try to write it as a casual message, the system ignores it.
The admin has unrestricted access. Free chat, direct commands, memory writes, configuration changes.
This sounds like a standard permission model. But the distinction that matters is not who can read or write. It is who can teach. Because every memory entry changes what the agent knows. And what the agent knows shapes every future output.
Why AI agent memory becomes a shared knowledge base
This is something I did not fully appreciate until I saw it in practice.
In the previous article, I described the structured memory layer, a file the agent reads before each run, containing lessons from past sessions. What I did not say is what happens when that memory becomes shared.
The moment multiple people contribute to the agent's memory, it stops being a personal tool. It becomes a shared knowledge base. Every entry affects every future session, not just for the person who wrote it, but for everyone who interacts with the agent.
Uncontrolled access to that memory is a real risk. Not because people have bad intentions. But because the agent does not distinguish between a well-considered methodological insight and a casual remark someone typed without thinking. It treats both as equal truths.
Command-only memory access for analysts was the compromise. You can contribute. But you do it deliberately, in a structured format, and it is logged.
What happens when the agent gets the methodology wrong
This one caught me off guard.
Join the Library
Full access to my findings, personal stories, and what I learn from the people I meet.
Join the Library · €29.99 per year How a small business, creator or politician can be… What should I study in the age of AI?… I Left My Job for Several Months to Read… How I Fell in Love With Her, at First… How do you stop depending on social media? Can… How to Start With AI in a Company: Automating…Get the full article by email and feel free to reply if you want to discuss it further.
Summary
Common questions on this article's topic
What is role-based access control for AI agents?
Why is AI agent memory a risk when shared across a team?
Can AI agents get trapped in logical errors?
What is the difference between an AI agent correcting itself and actually learning?
How do you deploy an AI agent for a team safely?
What is the biggest challenge when scaling AI agents from solo use to team use?
If you have any thoughts, questions, or feedback, feel free to drop me a message at mail@richardgolian.com.
LinkedIn
Related articles
In April, in the first part of this series, I wrote about an AI prediction system I had started building on my own machine. At the time the software was a few hours old and the prediction record was empty. The record since then has shown one thing: the system does not yet understand the market it is being asked to forecast. It can pull macro context, book value, earnings. But it cannot put those together into something that helps it understand the price.
I am building an AI system to predict the S&P 500. It runs on my own machine, uses free public data (yfinance, FRED, the Shiller dataset), and grades every forecast against reality. This series documents the build itself: the decisions, the methodology, the mistakes. What I will eventually share from the running system is a separate question, and an honest one.
Yesterday I could not tear myself away from the computer. When I lifted my head, it was half past eight in the evening. I had been sitting alone upstairs for about three hours.
More articles
Are you a content creator playing by the rules of YouTube, Instagram or Google? Are you a small business owner whose business model depends on how often Amazon recommends you, how expensive Facebook ad impressions will be or how expensive Google Ads clicks will be? If all or most of your contact with your audience or customers depends on a big platform showing you to them, you are not in a strong position.
The development of artificial intelligence changes how we look at which paths in life make sense. Which studies and which profession to choose.
It was Thursday, 31 March 2022, when I handed over the last of my responsibilities to my colleagues, closed my laptop and went to the library. I left my job for several months and immersed myself in philosophy. And I am immensely grateful for it. Today I am, like most people, on a merry-go-round, and it is hard for me to stop and think about the fundamental questions of my own life.
It was September 2023. My life, my work, everything was completely different from now. It was simpler. I owned almost nothing. I needed to film an advert, I went up to a floor inside the company where the product I needed for that advert was, and there I noticed her for the first time. She laughs at me when I describe it like this, but at that moment the universe told me, that is her.
How and why I am gradually leaving social media and the big platforms. When I read that Meta is testing a new way to charge for organic posts that link outside its platform, it confirms how right I was to become steadily less dependent on the big platforms and on social media. It does not mean I have left them completely. It means that whether or not I stay in touch with my readers depends on them less and less, and today almost not at all.
One of my family members described a firm that has its data in several different tools, and its employees still join it by hand, in spreadsheets. That is the ordinary state of firms. When somebody tells such people not to do it by hand and to use AI, they will not understand. Most firms do not even have the basic connectors between the tools they use. These firms need help with AI transformation.
The same task, two models. Fable 5 against Opus 4.8. On paper Fable is the better model, with a larger context and stronger specs. And still it lost. Opus handled the task with a single round of checking for 721,000 tokens, while Fable needed nine rounds and burnt through 2.78 million tokens. The difference was not in the model, but in how I set the task. And I know it, because I measured it.
A few weeks ago I installed a small local AI model on my laptop that watches a live camera feed. I turned the webcam on in the dark, and in near total darkness it recognised me and the objects in the room. That such things exist, I have known for a long time. What opened my eyes was the accessibility. I installed it in one prompt, free, and it runs entirely on my machine, sending data nowhere.
I once wrote about building my own privacy-friendly analytics tool. It had bot detection from the first version, yet it was not enough. Direct visits took a strangely high share of my traffic. When someone claims that 20% of their visits are bots and 80% are humans, I used to think the same. Today I would say the opposite ratio is closer to the truth. This is how I got there.
I have Heidegger and my notebook beside me. I am asking where all of this is heading, where artificial intelligence is taking us.
Four days in Catalonia. No computer, no AI, almost no social media. I bought this notebook so that I could write down what I would think about, and what I would come across and learn on the trip.