Przejdź do treści
← All English episodes
Opanuj.AI Podcast

How AI-Native Teams Work: Code Review, On-Call and Team Size

When agents write most of the code, practices built around human speed stop working: review queues grow, merge conflicts pile up and nights get noisier. Przemek Smyrdek and Marcin Czarkowski look at how Imprint, Anthropic and others are responding, with Will Larson's small change rule, layered automated checks and AI SRE agents that investigate incidents in parallel. They also ask whether two-pizza teams still make sense, and what smaller "two-slice" teams mean for ownership, pressure and wellbeing.

The episode audio is in Polish. The transcript below is an English translation.

Also available on

Sources discussed in this episode

Episode transcript

Automatically transcribed and translated from Polish - it may contain minor errors.

Back to the player ↑

Welcome to the Opanuj.AI podcast, where we analyze the most important changes in the world of LLMs, agents and generative AI. Your hosts, as always, are two experienced developers and agentic coding mentors, Przemek Smyrdek and Marcin Czarkowski. A warm welcome to all of you. In this episode, we’ll talk about what’s happening to processes in IT that for years were seen as untouchable best practices. We’ll talk about code review of every single line of code. We’ll talk about on-call shifts and teams of 6 to 10 people. All these processes, all these practices, were created assuming that the main bottleneck in IT is the human writing code. And now in many companies most of the code is written by agents, so that approach stops working. Code review obviously gets jammed up. We’ll use, among others, the example of Imprint, where they merge 4 times more PRs than 2 years ago, and the median review time went up from 33 to 77 minutes. Will Larson proposes the small change rule, and we’ll explain what that rule is about. Then we’ll move on to the well-known but disliked night fires and see how we can use AI agents in this SRE process, which will investigate incidents in parallel, and whether it’s worth building such solutions in-house or buying off-the-shelf ones. And we’ll talk about these teams, what they should look like now. Do we still want a team you can feed with two pizzas, or should we approach this a bit differently today? Przemek, where do we start here? What assumptions do we want to challenge in some way?

We should start with the backdrop that we’re basing this whole discussion on. Some assumptions that our industry operates on. We kind of have two worlds, you could say. The world of today, already almost yesterday, that we’re sure we know, that’s proven, we know where its edge cases, problems and challenges are, but overall it works. And then we have the world of tomorrow, where we invite agents into our environments and our teams. And of course there’s a certain tension between these two worlds. I’ll talk about that world of tomorrow for most of this episode, but we’ll start with the world of today, where, for example, code review is still mainly based on people, on their availability, on the assumption that people have time, have headspace, that they can process a certain number of pull requests that appear in front of them. We definitely have version control systems that today, in 99% of cases, boil down to git and the distributed model, where everyone clones the repository once and then works at their own pace, in their own reality, on that copy. and eventually they merge their changes. We’ve gotten used to that too. On-call shifts, which also largely depend on people being available, on how well we’ve configured notifications, whether we’re near our phone, or whether we have, I don’t know, a pager, some app like PagerDuty. And all of this is tuned to the pace at which humans produce code. To people’s availability and to the fact that, say, during a week you get I don’t know, 5, 10, 15 pull requests, because that’s roughly human throughput. And here’s the thing, there’s a challenge: that world pretty much no longer exists. The challenge is that this world is gone, and now there are companies that already know this and have accepted it and are trying to prepare for it. There are companies that feel this world is gone but don’t yet know what to do about it. And then there’s this third category, which we probably won’t really dive into today, but we’ll see if we go there: companies that completely deny it’s true or people who say it’s not true, just some fairy tales the guys on the podcast are telling. Let’s focus on what’s on the horizon. We’ll use Anthropic’s recommendations here as a kind of guide to what’s coming, because they write that a company working with frontier models is about 12–18 months ahead of the rest, who don’t yet feel what’s really hiding behind this potential of artificial intelligence or under this potential of artificial intelligence. So Anthropic, in a blog post in October, described a few points on how their workplace, their work environment is changing, and from there we’ll jump into a discussion of how the whole workplace in our companies might potentially change. First important point: by March 2026, agents had already worked more hours than employees, than the humans who were supposed to be steering those agents. Today, for every hour of human work there are about 6 hours of agent work, so roughly a 1-to-6 ratio. We’re seeing an incredible growth in merges across the whole version control system, in what has to happen at the version control layer. Anthropic says that in May they were merging about 2,500 changes a day, and in September, 30,000 times a day, so a 12x increase, some absurd numbers and stats. And of course, with that kind of growth, all those carefully designed processes, agreements, agreements tailored to humans just stop working. Anthropic notes—and you definitely feel this too—that review queues are growing, conflicts are increasing, our 10xDevs students tell us this as well, and there are more nighttime alerts, which also have to be managed somehow. And now there are different ways of dealing with this problem. There are companies that have hit this wall, think about it critically, accept the potential of these new tools and are trying to manage it somehow. And there are also companies that unfortunately still haven’t crossed over to the other side of the force. And like I’m saying, they either don’t use AI at all, or claim it’s all nonsense. Let’s ignore them and talk about how to deal with this new world, with this new reality where agents and people with agents are processing code in massive quantities. And we’ll start with what’s basically already a total meme, a potentially unsolvable problem, namely those infamous bottlenecks in code review. We write faster, but there’s no way to review it all. So what’s the point of all this AI then, Marcin?

It’s easy to say that what Anthropic is presenting is just marketing, or some kind of AI psychosis, that they’re overdoing it, but there are also plenty of companies that are, let’s say, closer to what we see in an average, including Polish, company. And I think a good example is what Will Larson talks about, an engineer as well as an experienced engineering manager, author of popular books and a great blog. And he describes how the review process is changing at Imprint. They have three times more PRs than two years ago. Maybe that’s not 30 thousand changes a day, but still, three times more. That already causes problems, and I think many of us deal with that in our day-to-day work. And despite everything, we’re also merging roughly a dozen or so more changes than before. Because of that, the time spent on review at least doubles, like in Imprint’s case. In many companies it’s probably much more, because many engineers weren’t really used to doing code review at all. It was often a duty delegated to that one person who was maybe less assertive or for some reason actually liked the process. And now that there are no excuses and everyone has to do it, this mid‑proxy pattern starts to appear, which people talk about a lot lately.

So I take a PR, paste it into my agent, ask for comments, ask for an analysis.

Exactly, and then we paste those comments back, without really understanding whether they make sense. And then that person throws those comments into their agent, checks if they make sense, and we end up with a conversation between agents, mediated by a human.

Someone recently gave it a great name: a slop grenade.

Yeah, these are also those slop grenades, it’s a disaster. It’s really hard to work like that. And what Will Larson showed nicely is that in this situation it’s worth asking ourselves, why we have a given process at all, what it’s for. And similarly we can ask: why do we do code review, why did we do it in the past, and think about how we can now keep the same benefits, still meet the same needs within the organization, maybe in a different way. And here Will Larson points to four main reasons, which I fully agree with. In our trainings too, we’ve also been pointing out these key benefits. There were quite extensive lessons in Opanuj Frontend about code review, how to do it well. And here, knowledge sharing is a key part of code review. Thanks to that, we improved our programming craft and also our understanding of the system itself. So how can we approach this now? By reviewing whole clusters of PRs and visualizing that knowledge, we don’t necessarily have to focus on line‑by‑line code review to reach the same goal. We also want to prevent bugs, of course. We don’t want what goes to production to be low quality. So again, we can lean on a better CI/CD pipeline, on better automated verification. We can work around these limitations that seemingly appear if we don’t do very detailed, line‑by‑line code review. Next, we don’t want to introduce any malicious changes that for one reason or another would end up in production. Whether it’s because of some library we unknowingly add to our code,

or an employee who’s leaving.

Who, yeah, wants to mess things up on the way out. So again, strict rules on what we ship to production and what we’re allowed to do. And auditing those changes both automatically and manually. And we also want what we deploy to be compliant. Especially if we work at a bigger company, constantly going through various audits. Then we have to be aware that there really are areas where human review is indispensable. If you work at a bank, if you’re modifying some critical payments module, then most likely all these optimizations aren’t something you’ll be able to apply in your day‑to‑day work. Because there are legal constraints you have to account for. So in the past, review covered these four different reasons. That was absolutely fine.

Let me interrupt you here. I’m wondering how many people consciously broke it down into as many as four dimensions. I think that really a lot of people who talk about these topics see code review only as representing one or maybe two of them. Often it boils down to “Did I see the code before it went to production, or did I not?”. But hardly anyone asks themselves why we want to see that code. So this is a great distinction of what can be going on, what the outcome is. You also do a lot of experimenting with knowledge sharing in general. You’re the kind of guy who’s constantly tinkering, looking for different ways to present what you have in your head. And this ex‑visualizer is a mechanism you’ve used, a skill. And I think it’s great in general that we sit down and think about why it’s even there. Maybe it shouldn’t even exist, maybe we should just ship to prod, but no. We can address different needs in different ways. The fact that we once had some approach that no longer works doesn’t mean we’ve now lost everything and we’re proverbially screwed. It really doesn’t have to be that way.

I also think a lot of people mainly associated code review with preventing bugs or maybe preventing tech debt. It also really depends on who’s doing that code review. So everyone has their own goals there.

Some sense of control.

Exactly. And so on, and so on. But that has kind of naturally fallen out of use, which doesn’t mean we can’t deal with it. So let’s focus on the sheer volume of these PRs. How can we change that? You talked about this in the latest episode on YouTube, which we recommend. What is the Small Change rule that Will Larson recommends all about?

So Small Change is based on the assumption that we have to automate something. Not everything. There’s some subset of pull requests that should go to production or towards production faster than before, or with less human involvement, basically. So now we ask ourselves, okay, what are the rules for a PR that can be automated, for the path of a PR that can be automated? And I’ll point out here, this is not a single recipe that we just copy everywhere, all the time, regardless of the company. It’s simply a variant of what Imprint is doing and what Will Larson writes about on his blog. So what are the criteria here for automating a pull request review? First, the scope of changes, which is defined here quite arbitrarily as simply a thousand changed lines of code. I saw, for example, in a code review discussion that for some people this is too much. For them it’s not a small change at all, it’s more of a medium change.

By old standards it’s a big change.

Exactly, but today we’re probably operating at a completely different scale, so those thousand changed lines of code are still treated as a small change. Probably because if an agent generates something under that thousand-line mark for you, it’s still basically a mini prompt, you could say, a mini tweak. So we have that scope. Second thing, we integrate the bot. We integrate some kind of agent as a judge, who just watches what’s going on, runs specific tests here, or some kind of security audit, and in Imprint this is defined with different error categories, or priorities, or some kind of impact. Here it’s labeled P1 and P2. Probably some critical bugs with high impact on that codebase. Those are not allowed. If this shows up, if the bot finds that kind of vulnerability, then we immediately exclude that pull request from this automated path. The author has to be trusted within that codebase and for that change they must have at least 10 merged PRs in that service. Also a great idea. I think the tip here is that this trust is super important and that experience within a given area of the company really does the job and is hugely valuable. The change touches fewer than N services, so you could say some kind of entropy, nicely put, how much chaos and confusion you’re bringing into this project. And we don’t touch shared libraries, because you can really mess up someone’s life, and we don’t want to do that. And those are the assumptions. Behind that there’s now specific tooling. We mentioned one kind of tooling, meaning a bot for code review, but we also have concrete skills here, or config files, that affect how this process works. In a sense we also have a kind of self-service, or simply the option to adapt these practices to your needs. So yes, at the project level we have a small change file that lets you turn off individual paths of this project from this process, based on certain sensitivity rules.

Very important when we have such critical modules that are very important for the system’s operation, then we can disable the small change heuristics there, so remember this is not purely binary. We have a lot of ways to tailor this to your needs and preferences. As Przemek said, the values that work at Imprint don’t have to work for you at all. And let’s also remember there can be many systems, for example with a small number of users or low risk, where you can even go a bit crazier.

Yeah, where it’s going to be hard, right?

Yeah.

Yeah, yeah, 100%, 100%. On that small change topic, I’ve got a question for you, since you’ve spent more time in platform teams. It’s basically a question, or rather a request for a comment. It’s a nice way of sharing these kinds of tools or frameworks, where you define the mechanism, but you give configuration to the end user. As the author of this change I don’t know where your critical paths are.

100%, that’s also very important when delivering this kind of solution for teams, to make sure you provide the right flexibility. Of course you need to set reasonable boundaries, ideally by talking with the teams and really understanding what’s critical and should remain unchanged, and where to allow flexibility. And that also depends on the company, on the scale, there’s no single right answer for how to do it well. This is where that famous engineering judgment and product thinking comes in, that we’ve been leaning into recently and we feel it’s still a big advantage and so‑called job security. I don’t feel that LLMs will be able to do this work instead of humans in the foreseeable future.

Exactly, I can imagine a prompt to an LLM saying make me a system like this, and the LLM says okay, I’ll wrap it all in some util, we won’t let anyone touch it, and that’s it. But no, that’s not how the dynamics in teams work.

It’s again a bit of a casino game, there’s a chance you’ll roll it just right, and often you won’t roll it right. Some people will say, oh, AI is brilliant, because what just happened to come up brings those kinds of insights to the table. That’s not the point at all; remember that those key decisions should stay on our side.

Exactly, so we have this config file. On top of that there are two helper skills. I assume that’s something this bot can use. Here we’ve got a pull request review, it can be either an approval or skipping that approval with no comments, there were also notes there about how a lot of that slop also shows up at the level of AI integration with pull requests, so we don’t want to generate that slop here. Review PR cluster is also an interesting skill, which, using automation or a bot, reviews batches of PRs and suggests topics for people to discuss, for example, what we often work with, what changes frequently, what to pay attention to. It’s great, you can bring it up during some planning, during a retro, you can interpret it in a hundred different ways. All of this is tied together by Fleet, which is such a central harness. We have here a kind of mechanism that checks whether all these checks are satisfied. Handed over, I think, to this fast integration path. And also, importantly, those changes from small change don’t bypass the classic CICD. Tests still have to run, and still probably some deploy to a test environment is performed and all the other methods we usually integrate into CICD to assess the quality of this pull request. And another important assumption: if a bug does slip through, we tweak this pipeline, instead of abandoning the whole concept. We just want to keep polishing it here, raising the quality, not backing out too early. Really interesting stuff.

What’s interesting, also interestingly, around 2016, in 2018, if I remember right, at Smart Decoder, where we worked together, there were already attempts at this kind of automation. I also kind of smile whenever I hear that nothing can be done about code review. If someone has never done anything and just accepted it as a golden rule of the universe, then sure, it can’t be done. Although I don’t know of any equation that says code review has to keep the shape it had in the past.

Exactly, so I think it’s a really valuable post, a valuable article, one that generally spurs you into action, to sit down and work on this. The question is, are there any caveats here, maybe let’s also talk a bit about the effects that it generates and how you see it overall.

You definitely have to assume there’ll be some kind of transition period, that it won’t work that well at first. We have to accept the risks and also the learning process, which is always demanding. You just need to experiment and it’s not worth giving up after the first or second failed attempt, because what’s really cool about programming with AI is that we can learn very fast. We quickly draw conclusions, and the next iteration doesn’t cost us another quarter of work, another week, it’s usually just a few hours. The only question is whether we have data, whether we have new takeaways, because otherwise we’ll just be wandering around, so we encourage a kind of lab-like approach here.

Like an engineer.

Exactly, like an engineer. We’ll talk more about that, by the way. Przemek, we also have new heuristics that Anthropic is using. Maybe let’s not go super deep into details, because it’s a similar story, just times 10, as usual with Anthropic, but they’re also using an interesting approach that you can adapt and factor into your own solutions.

Yes, and again here, you can go into the technicals, you can go into tactical solutions, but we mainly want to stress that this is a multi-layer discussion. It’s not like some change appears and it’s suddenly one inch from production.

And that’s just how it has to be.

That’s how it has to be, and it’s not just a toggle, ship or don’t ship, the real work starts earlier. Even, damn, on your own computer.

100%, 100%. And it’s not like Anthropic will tell you how to do this right in your team, they won’t tell you that either. 10xDevs, of course…

Let’s give you building blocks you can assemble yourselves.

Yes, exactly, because these are heuristics worth testing and seeing how applicable they are in our environment. Also remember there are lots of things that already worked before, for example, blocking automated checks. We can make it so that if tests don’t pass, the PR is blocked. An amazing solution has existed on GitHub for as long as I’ve been using it, which is quite a long time now.

Literally yesterday I saw a comment under a thread on LinkedIn saying, “Hey, you’re talking about code review, and people here don’t even have CI/CD yet.” That’s not really how it is overall, right? In general there’s a spectrum of whether you have CI/CD or you don’t. Plenty of companies don’t, plenty of companies are learning, and for many companies it’s something pretty trivial to which we’re just adding this AI block.

100%, so it’s also time for that more engineering-style level up. Let’s stop copying binaries onto some VPS without a CI-CD pipeline, because now that’s just asking for a ton of trouble. It was already a huge source of problems before; now it’s just more visible. I really like that observation that you’re only as good at AI as your platform is. You can kind of see now who’s been swimming without, you know, metaphorical underwear.

Without a platform.

Without a platform, so this is kind of the end of that honeymoon period, when it was easy to ship software and make money on it. In a way that was a bit non-engineering, that’s the truth. So it’s also high time for software work to become actual engineering, and not just some job of cranking out code based on tickets.

The six layers of control that Anthropic talks about. First is local verification. We have agents. Those agents, first and foremost, are still mostly associated with something that runs on your own computer. We work with cloud agents. I’m not saying this to brag or anything, but it looks to me like that’s not the market standard. This concept of a virtual entity living somewhere in CI-CD is not a standard at all. We hear that companies are experimenting with it, but we’re only just getting used to it. So everything starts on your machine, you could call it a quality shift left. When we have changes on a branch, changes in some work tree, you can ask the agent what it thinks about them, whether it’s okay or not. Ideally, you standardize this within the team. Once it goes out, once it lands on some path into CI-CD, we have a pull request, so we do an AI review. And again, some standards here: agreeing on a definition of ready, or basically a definition of done. Again something we mentioned and highlighted, how many teams have never actually sat down and talked through what it means for a PR to be ready for review. Or what it means for a PR to be ready to go to production. Security review, different kinds of audits. Automatic blocking checks, both the ones like a security review, and the code-related checks, some kind of static analysis. Here as well you can deterministically specify those rules with some config, some eslint, and so on, and so on. Human approval, or generally human involvement depending on the risk level. We again evaluate it based on some heuristics, what kind of change we’re talking about. Here you could probably either take the number of lines of code into account, or the area of the system the change touches. The services it affects and so on, and so on. From 0 people up to some set number for the heaviest changes, the ones with the highest impact on our project, you could say. And the deployment process itself, merging all these changes in. It’s not like we just ship straight to prod. Anthropic here talks about very heavy use of feature flags, switches that simply control the availability of a given change, a given feature. And also these sort of adoption rings, so to speak. First internal users, then the latest channel, meaning the early adopters, and finally the stable channel. In Chrome, for example, we have canary, nightly builds and so on, and so on. So you can really draw from so many areas of software engineering here that it’s actually fun to talk about.

True, and I’d once again emphasize that element of local verification, but which should also be done in the cloud, based on static code analysis. Let’s not ask the agent whether this is good code. Even if we feed it specific guidelines, because 99% of those can be automated using all kinds of linters and formatters, really eslint and similar tools. They can tell us everything about whether the code is high quality, aside from really minor exceptions. If our review with the agent boils down to feeding it the linter rules word-for-word and hoping that this way we’ll be safe, that’s not what this is about.

This is probably especially important if you didn’t have any such control mechanism at all before. In a way this is probably an extreme case I’m talking about. A bit of an exaggeration.

Or maybe not.

But what I’m saying is that jumping straight into AI in this area is super risky. It should rather be an add-on to what already works. If you’ve got nothing and you’re starting from this kind of…

This is like step five.

Yes, this is literally step five. And again, from my experience most of the code review I took part in in the past was basically just a human linter. Why did you use var instead of const?

Nitpick colon.

Yeah, and if there even was that nitpick colon, God willing. And that’s what our brilliant code reviews we were doing looked like. And we’d spend three days arguing whether this class should be called subscriber, observer, singleton, factory or maybe the order of those nouns should be different.

You can also codify that in a linter, if we consider it important. In my opinion it’s not particularly important, but here everyone can decide what they’d like to have in their own codebase and what they wouldn’t.

I mean, that should just light up on your machine.

Yeah.

It shouldn’t be a topic of discussion during code review. Same as, I’m thinking, compliance, at least in our webdev world, compliance with some browser APIs and so on. There are adapters and so on, or linters, or even just the fact that it all gets prepared properly after the build. We have to separate this, we have to remember that. Recently I had this 10xDevs feedback interview, where one of our students said that it’s great you point out there’s a whole bunch of classic tools that you only then integrate with AI, instead of starting from AI and having everything else just tacked on.

Right, and that’s again this kind of experience, a bit of awareness of what software engineering work actually is. And paradoxically it also shows that we’re not actually that thrilled with AI in those particular areas, in that literal sense of flipping a switch and using only that.

I actually feel that’s when people think we’re thrilled with AI. A lot of people assume that’s how it is, and then they come to our trainings and we just talk about the same things we talked about in trainings even before AI was introduced, we just show: look, you have this, and now add this amazing technology on top, which gives you leverage and lets you focus on all the important things, that don’t rely on cranking out little bits of code, because we’ve had this kind of bold idea for seven, eight years now, that building technology was never really about just writing code for its own sake. In this Anthropic story we’ve got one example of how two agents can be connected, one for generating code, the other for reviewing code. There was one of the screwups here that we can talk about too, Marcin. How was it with those night-long rounds of that whole review, where the agents got stuck in a loop?

That’s exactly what you need to watch out for when you build any kind of para-family of agents, as I like to call it, that sit there talking to each other.

It’s already a classic, this para-family of agents.

You need to assume that such a para-family can get into a close relationship and talk to each other for a very long time without really contributing anything to your company except a growing API bill. So at Anthropic there were also cases like that, with two agents, one verifying, one reviewing, and it went on for a very long time, all night.

50 rounds.

50 rounds, an enormous amount, so it’s worth also… If we’re building this kind of solution, it can absolutely make sense, although we recommend starting with the simplest possible setups, and only making them more complex if the simple one doesn’t perform the way we’d like. So let’s cap the number of turns that can happen between these agents. Here the suggestion was, for example, 3 turns, and if the agents don’t reach anything by then, then a human steps into the game.

I think it’s even worth distinguishing a classic agent-style workflow, where something runs in a loop, from a regular prompt we just send to the API. Because in code review you can easily imagine we have a prompt with some system instruction, we know the diff, we get an answer, and that could be the end of it. It doesn’t have to be a loop of “I’ll keep fixing this forever.” You can have a different agent use case called “get this to green,” so to speak, but again, those are two separate matters.

What’s important and relevant in this para-family pattern is separating context between the agent that verifies something and the one that fixes it. Because if one agent does both, it will naturally spot fewer issues in its own work. So that part is actually a plus, which is why it’s worth experimenting with it. Just again, start from the simplest solutions. When after the first 30 minutes of work you see a 30-node graph of your system, you can feel good about it and very professional, although it can also lead to high costs, complexity, and not really achieving the goals we actually care about.

Again you get this effect that in these graphs, the companies that consciously map out dependencies between services, for things like contract tests and so on, get an amazing leverage, that edge, and companies that do it just to post a screenshot on LinkedIn, or people who do it just to post a screenshot on LinkedIn, are once again just doing theater.

So you really need to clearly define your goals somewhere. Of course different units have different goals. Remember that if we’re in a company, in a team, our goal is an efficient SDLC process, and not so much for something to look cool so we can brag about it in meetings or on social media.

I’ll say more about these effects at Claude Code, what actually changed. After they gradually improved that process, those rejected PRs, that weren’t at all ready to go to production and were just wasting our time, their share dropped by 40%. So fantastic, 40% of unnecessary PRs that nobody has to review, because they’re filtered out earlier by all kinds of automation. That’s great, right? That’s what we want. Yes, it’s awesome. The share of internal PRs with substantive comments went up from 16 to 54%. Also great, right? More thoughtful communication, more knowledge sharing too. Here Boris Cherny, also the creator of Claude Code, says that when Claude has to check its own work, assuming, of course, that the pipeline is set up properly, the result is 2–3 times better compared to, say, a quick human review, for the volume of changes we’re talking about, or compared to when we do it carelessly.

That’s also something people keep repeating more and more often, that automatic verification is very important, and I fully agree with that. An agent, just like a human, needs feedback. We often relied on manual tests, or on our experience and intuition. An agent has neither of those, so you have to give it other ways, and there are really many of them. These tools for automatic verification have been developing nicely lately.

All right, so we have a change, sometimes a small change, sometimes not at all. We’ve improved the code review process, we have extra integrations, we go to prod. It reaches the target group of users, but something may go wrong, as it usually does.

Sooner or later it always does.

Sooner or later something goes wrong, so now we want to move to the topic of fires and how to put out those fires in the age of agents. Literally yesterday, on October 27, a very interesting article came out on Gergely Orosz’s Pragmatic Engineer blog about AI SRE, which shows that again there’s a slightly different angle here, about where to integrate these agents. We’re no longer talking about changes, about the moment when something hits production, we’re talking about a situation where something’s already in production and it’s starting to burn. Obviously, when we were putting out that fire only depending on people’s availability, the results were… mixed. The speed and efficiency of that process, the way we kept repeating the same mistakes, whether we were really addressing the root cause or just treating some temporary symptom, parties, sickness, vacations. I think there are plenty of situations where SRE was really put to the test, and then there’s also this whole concept of bus factor, how many people actually know how to fix a given area, or a given model, or a given service. It’s really a major risk, directly affecting end users and also the production environment. It’s something we really need to deal with. And on the Pragmatic Engineer blog there are a few ideas here that we’ll talk about in a moment. First, an observation that the number of incidents that can potentially appear in your system comes from two components. First, the sheer pace of change: if we speed up, more of them will appear, but it’s also multiplied by the probability that a single change will break something. And now, depending on how robust and resilient to change your system is, that pace is either a small problem, a medium one, or a big one, or might not be a problem… although “not a problem at all” probably doesn’t happen.

Inevitably it’s a factor in this equation, so you have to accept that if we ship more to production, we’ll most likely have a higher number of incidents. How much that number goes up obviously depends on a lot of other factors, which are hidden, because this is a pretty simple equation, but a very accurate one. Again, simply because there are more incidents. There are various reports that prove this. It’s a kind of software engineering law.

It’s basically a truism that if we ship a lot to prod, we’ll get more alerts. It’s like: the more you ship, the more lines of code you have in production. Thanks for the info. Usually, that is, so nobody nitpicks me: unless your change deletes lines of code, but then hats off for that observation.

Exactly. Anyone who’s ever configured alerting in the good old days knows it’s extremely hard. It’s hard to have automatic alerts that give us a clean signal without noise. Inevitably we either have too few alerts or too many, and in most companies we still found out about a real outage unfortunately from customers or from our director, who would call us at 3 a.m. saying production is on fire. Now I’m sure people from those so‑called unicorns will show up again, although I’m not talking about valuation here, but those exceptional companies where it supposedly never happens. I never had the privilege of working at such a company and I also haven’t heard about one first‑hand, although undoubtedly there are tons of them, it’s just that we keep saying this is all so hard.

Although it seems to me that among our developers, in our industry, the agreement on this point, that there are bugs in production, is pretty strong. Here it’s actually a bit easier to talk about it, because we know stuff gets messed up.

That’s true, that’s true. And it’s really hard to find out about it very quickly. It’s just difficult, but I think that in these times of agents there’s huge potential to find out faster, not from customers, because it’s a more intelligent system than those old if statements that automatic alerts have basically relied on so far. So now, Przemek, how can we do this today? What’s the current approach?

We can approach it by giving agents access to all kinds of observability tools. We have Sentry, we have Cloudflare, all sorts of solutions where we log errors, observe traffic, and then create more flexible heuristics that will tell us, damn, it started throwing a bunch of 500s here. But maybe having 10 500s over two minutes still isn’t a reason yet, right? Let it look into those Sentry logs to see if it’s actually throwing something important, if something is really happening here. So we have many more options to combine different information sources and also to draw conclusions in a more dynamic way. Again, configuring this to be reliable is very hard, although there’s a lot of potential. We’re also experimenting with this in our systems now. It gives mixed results, sometimes better, sometimes worse, but as someone who’s spent time in Grafana and Kibana, well, it doesn’t seem to be, most developers’ favorite experience, so here I feel there’s a lot of gold to be mined.

I’d flip it around from the perspective of motivation, of why you’d even bother with this at all. You don’t want surprises, you don’t want to lose your reputation, you want to react faster, react automatically, but not just out of your own convenience, you just want to deliver that quality in this way. And not at three in the morning when we’re asleep somewhere, but it’s better to find out about it during the day, having better insight into how the changes that we’re shipping to production more and more often affect the system, so you can quickly roll them back, adjust them, and calmly close the laptop after four or five p.m.

Exactly. We have this timeline here that shows up in the context of a sample vendor here, Resolve AI, who rebuilt the kind of architecture we’re talking about, a notification architecture to handle reliability. In 2024 still… there’s a scripted workflow full of those if-elses you mentioned. Whenever we had even a slight mismatch with the scenarios, of course things leaked through. In 2025 it’s a single agent with a knowledge graph, getting lost in long context, understanding intent and basically interpreting all those different signals. In 2026 it’s like a para-family, but a para-family that’s already built on more stable foundations. Different agent roles, collaboration between those agents. I think 2027 and 2028 will also bring upgrades here, because this is a kind of journey. We’re not talking about some end state where you build it and forget it. You just set yourself a North Star, move in that direction, and discover the next chapters in the journey.

Yes, and now agents first of all have enough capability in terms of zooming in and out, speed, as well as access to all kinds of services. All the providers of top observability and alerting tools already have their CLIs and APIs, they have their MCPs that we can plug these agents into. A nice pattern for how you can distribute responsibilities when building agent-based alerting tooling, is this split where one agent observes and watches whether anything is happening, and one is responsible for communication with the channels developers watch, whether that’s Slack or any other solutions. Routing. Routing, exactly. And also a verifier that makes sure we really have a problem before that message gets sent. This is of course just one of many approaches. It’s worth experimenting here. It’s a nice starting point. And probably in the coming years, like you said, a lot will be discovered in this area.

If you had to name one action you can take to get that 5% improvement in your system, based exactly on what you did in our platform, because for everyone listening, it wasn’t like Marcin disappeared for four weeks and was gone saying, I’m going off to do observability to make this happen, we were still working on 10xDevs, doing support, doing the podcast, content, sharing this knowledge. You disappeared for a day and a half, that info showed up on Slack, but if you were to do a kind of just‑in‑time retro for yourself, where would you start, in your opinion?

First of all, I looked at what hadn’t worked in the past, where we most often slipped up, where problems appeared, what kinds of changes often cause problems, what information channels we have, and whether information there is being collected properly. Usually it’s not, usually our tools like Sentry are not tuned that well. Together with an agent we can improve that, detect what gaps we have. It’s now very easy to do. So first I would run a system audit: are we actually collecting enough of the right information? And also in retros, where we had those 3 a.m. wake‑ups and how we can prevent them.

Absolutely, I think Slack itself or similar communicators are also great here from the reaction standpoint, because you have a whole actions API there, buttons, you can again plug in all kinds of hooks on the other side that react. So the developer experience is really nice and friendly too. I think a lot of people who’ve built some automation on top of Slack know this concept of interactive messages. For people who don’t know it, basically it’s not like today on Slack you just get some SMS‑style alert and only then start working. No, you can have, for example, ABC buttons there: ignore, take an automatic action, or report, etc. And you’ve already implemented this too.

Yes, it’s very easy to add something like that at the pipeline level. Slack has a ready‑made API for it. I don’t know how it is in Teams; if you use it, my condolences. Maybe something like that works there too. It’s worth checking whether it exists.

Yes, exactly. On Discord as well, their integrations are very good.

Yes, Discord definitely leads in these automations.

So there you can also configure even more things than with Slack.

So, maybe let’s move on now to the last topic of this episode: how to adapt all this to the team structure, because that also raises a lot of doubts and the current state of things creates problems. Przemek, how can we open this topic?

Well, we have to drop by a pizzeria. We need to go to a pizzeria. We have to think about how many pizzas to buy so it’s enough for the number of people who are supposed to handle things. Some background again. I actually got interested in this topic thanks to one of the participants of 10xDevs, who asked us this question during our coaching hours, those Friday sessions where we talk about practice, who asked about merge conflicts. Conflicts that appear when everyone gets access to AI, we’re working densely, very close together within a project, and everyone fires up not just one agent, but hordes of their own agents. As you nicely put it on your LinkedIn: I’ll do the button, you do the API and TS, I’ll take border-radius, you take color.

Yeah, yeah, yeah, let’s roll with that.

Let’s roll with that. And generally, these aren’t really the times anymore when you can work like that. So in general, a lot of companies that approach this whole topic of organizing teams consciously, and rolling out agents, are thinking through the problem of the optimal team shape. We had Jeff Bezos, who quite a few years ago said that we should really have teams you can feed with two pizzas, so around 6, 10, 12 people, because that was the right balance between impact and, you could say, autonomy, and having all the roles on board, so there are no dependencies on other units in the organization. Only, well, the problem is that those 6 to 10 to 12 people are a bit too many for today’s conditions. Looking at how maintenance or development of various services or products in companies usually works, it generally seems we’re working too densely. We’re packed in too tightly, right?

At the level of the codebase. Between us it’s easy to see, because you could say each of us could be the owner of the whole platform, in a sense, as it evolves. It kind of naturally worked out that, for example, you took the user-facing part, and I took the whole CMS, and you mostly go into that CMS as read-only, and I go read-only into lessons and authentication and so on and so on. If we were working at the level of a single button, we’d have a problem here.

So more or less from this year, you could say, based on our research, Dan Shipper from Every was the first to post about this. Since the beginning of the year people have been experimenting with this concept of not two-pizza teams anymore, so not those 6–10 people, but rather two-slice teams, like two people for a given service, or even one person for some mini-product we’re developing here, who spin up their agents and suddenly scale up—their impact scales up. So we’re seeing teams that in the old world wouldn’t really look like teams, but in the new world they’re still effective, they avoid those common conflicts, but they do shift the bulk of the work onto agents. In our Polish backyard, Maciej Mendrela and the Kraków-based edrone are experimenting with this. Maciek also has a really cool, forward-thinking approach to integrating with AI, they’re not afraid of change over there at edrone. There was a talk about this on a live with the guys from AI Product Heroes, you can watch it on YouTube. So this challenge is emerging around exactly the shape of the team, how to work, in what configuration, so that agents’ changes don’t clash with each other, and the tendency is that teams will rather be smaller than larger. Do you see any possibly negative consequences here from a developer’s point of view, who hears something like this, if we were to grill it a bit?

It can definitely trigger the fear that if teams are smaller, then there’ll be fewer people in the company overall. But that’s again thinking in terms of correlation, not causation. You were talking about this recently, about what might happen with work.

Yes, there might be the same amount of work, or even more. We’re talking about team structure, meaning if we currently have a six-person team, we just split it, for example, into three two-person teams, where everyone gains full ownership of maybe one of those three modules. You know, it’s a convenient split of what the team has handled so far. And that small team is much more immersed in that one specific application, instead of having their attention diluted across three different ones, where we’ve got a ton of PRs and we’re no longer able to keep on top of everything. So we narrow our focus a bit and in return, of course, we take on full responsibility for what happens in that given module.

I’m wondering how we compensate for two things. Personal well-being, because that’s not obvious to me at all.

Not to me either.

That’s exactly the weight you carry on your shoulders. Because there will be more of it. And now the question is, how do we deal with that? Sorry, you’ve been a manager, so you also have some experience here. Share your thoughts, I’m sure you’ll be able to say something on both fronts.

I think it really comes down to personality, and it’s hard to generalize. I was more the kind of guy who liked delivering and having that space. I looked for chances to bond with people outside of what I was doing. That wasn’t a tool or a goal in itself for me.

I’m the opposite again, as we know. I really value that collaboration. And that’s an important element for me. I really wouldn’t want to be permanently on a one-person team. It’s super hard in general, long-term, to work four years like that in a company. I’d end up in a place that wouldn’t exactly put a smile on my face. For me, the more people I have regular, ongoing contact with, the better, so that has to be factored in. I think the importance of events like brown bags, demos, these team-bonding events really goes up. The importance of all this grows, because we still need that space to talk about what’s going on. Also from experience, if you don’t close everyone’s laptop lids in the team and say listen, we’re going to go talk now, and not have everyone staring at their own agent, it just won’t happen. Even if you have some description on a pull request and call that knowledge sharing in general, it probably doesn’t fully do the job. I think you really need to sit down and have that space to talk. This actually looks really good at Pragmatic Coders. I had a discussion with Wiktor Żołnowski at 10xDevs. He said that sprint planning or weekly planning is this kind of team event that kind of grows into something almost sacred. We turn off those agents and talk about what we actually want to do. I think that will somehow compensate for what’s happening in the day-to-day work. I find that crucial.

A great example of how you can do this even in a very large company is what Josh Woodward, VP of Google Labs, talked about, whom I had the chance to at least sort of indirectly meet at Google.

Our colleague, your colleague.

That’s a bold statement. Although we were in the same room a few times over the course of three days.

Demis Hassabis, also your friend.

Also, also. I can kind of say the same there. But what’s interesting about Josh is that he radiates really positive energy. He’s a very energetic person. You can see he’s really into this stuff. Despite such a high position in such a big company. And he deeply believes that small autonomous teams that have full ownership can do amazing things. He also talked about it on Lenny Rachitsky’s podcast. I hope I didn’t completely butcher that name here.

Lenny’s Podcast.

And in that podcast he talked about the behind-the-scenes of creating NotebookLM. Which is one of Google’s bigger successes in the age of AI. And that also came from a team that had no task like “go build this exact thing”. Just “experiment with AI”. And with how AI can be used for effective learning that will be engaging and interesting. And there were those famous podcasts that caused an incredible sensation two years ago.

Podcast Overview.

Yes, exactly. And his employees were the ones who created that as well. Two of them. And you have to listen to it. You’ll see if you like it. And he was completely blown away by what he saw.

And they were too, right?

They were too. That was the motivation. But precisely because it wasn’t a 12-person team with three extra layers of management on top. Just people who are insanely good at their job, have lots of room to operate. And thanks to that they can do incredible things. So that’s the upside here. Why this big autonomy, owning things 100% and delivering as a small, agile team can really give great results these days.

But still, it seems to me there has to be some compensation for that, some teamwork. That team building. He also says there that at Google they keep thinking about, for example, whether they should group these teams into some kind of clusters. Even though each one is this two‑person team or we become these two‑person teams, we’re part of them, whether we still shouldn’t, in sixes, go out for those breakfasts. I’m kind of adding that part myself. Or have one more event just to chat about things. I keep wondering what it’ll be like with this pressure. Recently I was listening to a podcast, maybe I won’t name names here, a person who’s considered an authority and who in my opinion is very wrong claiming that the LLM will take over the pressure and that the stress, the burden will be on the LLM, because it does the dirty work. I think that ownership will still land on the human. And we can already feel it, plain and simple. And I’m thinking, something that flashed through my mind here, for example cutting off access to the agent at 6 p.m.

At 4:01 p.m. I don’t know what podcast that is, or maybe I don’t know who you’re referring to. To me that’s total bullshit.

When I tell you the name after we finish recording, you’ll be surprised who said that.

You think I’ll be surprised? Or maybe I won’t?

No, I think you will.

Ah, okay, okay. All right. No one is going to call the LLM at 3 a.m. when the system goes down. These are the kinds of statements that show that someone probably has an interest in talking that way. Or they’re far removed from that kind of digital relationship. Or they don’t really realize what an engineer’s daily work looks like. That personally triggers me. I also like to say and write triggering things.

That’s the second scenario. Someone who used to be an authority, but is now a bit removed from the everyday reality, and a person from across the ocean.

Okay. I’m aware that sometimes I might write or say triggering things. Although I still try to do it in an empathetic way, to open people’s minds more and help them look at things more broadly.

I mean, that kind of oversimplification.

Yeah, yeah, yeah. Because I often get accused of that, but in my opinion it’s such an obvious oversimplification. So that’s minus one from me for takes like that. So, a lot is happening in this programming world because of AI. I think we can slowly start wrapping up. We’ve covered the most important topics. What’s the takeaway or final takeaways from this conversation? It’s worth forming your own opinion.

In my view, the first thing is that the old world is gone. The only question is whether you already agree with that, whether you feel it, or not yet.

Of course, I think everyone feels it. Many people react differently to the emotions they’re feeling. Denial and rejection are quite common reactions to big, uncomfortable change. Especially when we’re also worried about whether we’re not sawing off the branch we’re sitting on, as many people say. I get that.

Sure.

It’s a rational approach. It just seems to me unproductive in the long run, because over the course of history no one has ever beaten technological progress. To me it’s more reasonable to influence the direction it’s going in and position ourselves as well as possible in relation to what’s actually changing.

The second thing is definitely an audit of how we do things. And realizing—we have that “Talk to your agent” meme. Realizing that when you switch on AI code generation, everything else also has to be connected to AI in some way. There’s just no way you’ll turn it on in one area and the rest of the process will stay the same, because it’ll just fall apart.

That’s true. And let’s also remember that an LLM is not just a code-generation machine. It also has various analytical capabilities we can use to better understand how we do things and what we can improve. So let’s remember it’s not just a feature-producing machine, but also something that can help us analyze our code review process, how we used to do it, whether it was such a brilliant process, or maybe we were nitpicking about things that didn’t really matter and we have to do way more work than we’d actually like to, and we’d been running away from it for a long, long time.

And I think the third outcome, the third conclusion from all this is that there really is no old world anymore, but there are no sacred cows either. You have to challenge basically everything that’s going on and really, when a problem appears, paradoxically you’re in a position to position yourself in the organization as advantageously as possible, because you feel that problem and can decide whether it should be solved with an LLM or not, but the very fact that you noticed something is starting to crack at the seams somewhere is a great starting point for a new roadmap, for example.

100%. And also, not to take a completely opposite approach to what you said, I think there are some sacred things, and you definitely recognize them too, meaning the user should be happy.

That people should feel good at work.

That people should feel good at work and the company should keep doing well. Those are the sacred things. That’s what our programming work has always been about. And everything else should really just be a consequence of those things, which can change at the level of processes and our approach along with the development of different technologies. The leading one right now is AI agents, and let’s say the whole world is aligning to that. The way we’ve done things for the last 10, 15, 20, 30 years is not any kind of sacred thing in itself. It’s often just the result of what we adapted to. We used to adapt to the fact that we had few developers who wrote code slowly. Now we have a few more developers, because their numbers have grown. Plus, writing code has stopped being a bottleneck in itself, and because of that a lot has to change.

That’s a very valuable untangling of my simplification. Here, plus one, plus one for you. All right, that’s where we’ll wrap up today’s episode. As you can see, this is a special episode anyway, because we’re recording it in completely new conditions. We want to invest more in this podcast. We want the quality here to keep going up, so we’re recording this in Katowice, at the Daj Głos studio. We hope you’ll like this format. We encourage you to follow, to share it with others, to click five stars on whatever platform you’re listening to this podcast on, and we also encourage you to subscribe to the Opanuj.AI newsletter, every Monday, three hand‑picked materials about artificial intelligence and machine learning that we keep selecting for you ourselves. They’re connected with the episodes, with what we’re doing as part of Przeprogramowani and 10xDevs. It’s like a supplement, an add‑on to what you hear about here.

Exactly, we’ll probably be investing more in YouTube now too. So far we’ve been very focused on Spotify, so if you’re listening to us on YouTube, which we hope you are, then hit subscribe and the like button, boost this podcast so we can reach a wider audience interested in substantive conversations about agents and programming. Tune in for the next episode, which will most likely drop in less than a month. Spoiler. We’ll be diving more into the technical topics here. Today was maybe a bit high‑level, although I still think we weren’t just waffling, but there will be even more code and maybe it’ll be a bit more for the other side of the coin. Today we talked about AI almost only in superlatives, that everything is awesome and we should change our processes, and it’ll be more about what to watch out for, that it’s not all as simple as it might seem. We’ve also said many times, and mocked those takes, that you can learn AI in one day. So there will be a whole episode about the fact that you can’t, so we encourage you to listen. Thanks for being with us. See you in the next episode.