Rendered at 19:59:18 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
superfrank 2 hours ago [-]
Maybe I'm missing something obvious, but if I was going to have my team use this, I'd want someone to answer the following question
If AI is good enough to explain what the change is and call out what to focus on in the review, then why isn't AI good enough to just do the review itself?
I understand that the goal of this is to ensure there's still a human in the review cycle, but the problem I see is that suggestions will quickly turn into todo lists. Devs will read the summary, look at the what to review section, and stop reviewing code outside of things called out in the what to focus on section. If that's true, it means customers need to be able to trust that the AI has enough context to generate accurate summaries and suggestions. If the AI is able to generate accurate summaries and suggestions, then why can't we trust it to just do the review itself?
I'm not saying that to shit on the product, because I do get the logic behind it, but I think that's a question you should have a prepared answer for since I feel like I can't be the only one thinking that.
millbj92 3 minutes ago [-]
One thing that comes to mind is that an AI might see the code and say "Yeah, this should compile / no obvious runtime errors", but the AI doesn't have the context to know your teams coding standards (every team has different standards). That said, there are ways to feed that context to the AI, but still risk hallucinations, etc.
nijave 1 hours ago [-]
Imo human review is important for context/knowledge sharing even if a machine or tool can mechanically determine the change is reasonable
cpan22 1 hours ago [-]
Yep, for me personally, code review was the most effective way for me to get up to speed when joining a new engineering team
cpan22 2 hours ago [-]
No worries at all, that's a very fair point and a question we've gotten a lot!
I think our perspective is that: software design has always had a subjective element to it. There's never been a "right" way to design a system, there are always trade offs that have to be made that depend on things like business context etc.
To that extent, most engineers probably still want to be part of that decision making process and not just let agents make all the high level decisions, especially if they're responsible for the code that ultimately gets merged
embedding-shape 5 hours ago [-]
It's an interesting idea, but I feel like it's missing almost the most important thing; the context of the change itself. When I review a change, it's almost never just about the actual code changes, but reviewing it in the context of what was initially asked, and how it relates to that.
Your solution here seems to exclusively surface "what" changes, but it's impossible for me to know if it's right or not, unless I also see the "how" first and/or together with the change itself. So the same problem remains, except instead of reviewing in git/GitHub/gerrit + figure out the documents/resources that lays out the task itself, I still have to switch and confirm things between the two.
cpan22 5 hours ago [-]
I agree, that's also really important and something we're brainstorming
Currently on Stage we also generate a PR summary next to the chapters and that's where we want to do more "why" that pulls in context from Linear, etc.
And I know there's a lot of cool teams like Mesa and Entire working on embedding agent context into git history itself so that could an interesting area to explore as well
nathannaveen 5 hours ago [-]
I assume this problem could be solved if we write up what we actually want (like a GH issue) and maybe in the future the guys at Stage could use github issues as part of their PR review?
cpan22 1 hours ago [-]
Yep! Or Linear, etc
Or could be something like git-ai which captures agent context in git commits
Peritract 3 hours ago [-]
> more and more engineers are merging changes that they don't really understand
You cannot solve this problem by adding more AI on top. If lack of understanding is the problem, moving people even further away will only worsen the situation.
cpan22 3 hours ago [-]
I agree, and that's why we're not building a code review bot which aims to take humans out of the loop
We don't think of Stage as moving people further away from code review, but rather using AI to guide human attention through the review process itself
Peritract 3 hours ago [-]
Nobody thought of the other stages as that either. It still happened.
AI guiding human attention means that humans aren't guiding human attention, which means less human understanding of their reviews.
forthwall 17 minutes ago [-]
Interesting app, I have a weird bug I'm seeing with the homepage, when I tab between the chapters, it lags a bit then doesn't actually proceed to the next chapter until I press again
high_priest 50 minutes ago [-]
No pricing page, you've lost my interest. Doesn't matter that there is an obscured quote on the front page. Be up front about the costs.
cpan22 44 minutes ago [-]
Totally fair, we're working on it!
baldai 1 hours ago [-]
I was actually recently thinking about similar idea. I am someone who started coding post LLMs and have basic technical understanding. I know what loops, variables, API, backend bla bla is. I learned bunch more since then but I am not capable of making decisions based on git diff alone. And I want to. I want to because I think increasing my skills is still super important, even in AI era. The models are getting better, but are still limited by their core design -- for now it does not seem like they will replace humans.
So getting assistance in the review, in making the decisions and giving me more clarity feels interesting.
Maybe its people like me, who became involved into coding after the LLMs who might be your niche.
One thing I dont understand, the UI/UX? Is this visible only on git itself? Or can I get it working in Codex?
cpan22 1 hours ago [-]
Yeah this is a really interesting perspective!
We've wondered about what the review experience should look like for newly technical or non-technical people now that they are increasingly putting up PRs themselves. These people will be less opinionated about certain technical decisions in general so maybe the future looks like review processes very personalized to your experience level and your background. Definitely a lot to think about
Right now the chapters UI is only available on our website but we're exploring possible integrations and/or a desktop app
jFriedensreich 4 hours ago [-]
Looks kind of neat like devon.ai review / reviewstack crossover. But as i tell every of the dozens projects trying to make a commercial review tool: i would rather spend a week vibe copying this than onboarding a tool i have to pay for and am at the mercy of whoever made it. Its just over for selling saas tools like this. For agents i also need this local not on someones cloud. Its just a matter of time until someone does it.
cpan22 4 hours ago [-]
Thanks for the feedback! re: local vs cloud, I do think there is a cool work to be done around unifying the writing/reviewing experience locally, but we started with cloud because we designed this as a collaborative product with teams in mind
tasuki 6 hours ago [-]
> Stage automatically analyzes the diff, clusters related changes, and generates chapters.
Isn't that what commits are for? I see no reason for adding this as an after-thought. If the committers (whether human or LLM) are well-behaved, this info is already available in the PR.
dean_stratakos 5 hours ago [-]
In our experience, it's difficult to create well-mannered commits as you code and new ideas pop into your head or you iterate on different designs (even for LLMs). One concept we toyed around with was telling an LLM to re-do a branch using "perfect commits" right before putting up a PR. But even then you might discover new edge cases and have to tack them on as additional commits.
We thought git wasn't the right level of abstraction and decided to tackle things at the PR level instead. Curious to hear your experiences!
tasuki 2 hours ago [-]
> In our experience, it's difficult to create well-mannered commits
Sure, it is. But it's worth it, not just for code review, but for a myriad other things: bisect, blame, log, etc.
Your tool makes one thing (the code review) easier, while decreasing people's motivation to make well-mannered commits, thus making everything else (bisect etc) worse.
I'm sure it's net positive in some cases, and I think it's net negative in other cases.
tasuki 3 hours ago [-]
> But even then you might discover new edge cases and have to tack them on as additional commits.
Have you heard about `rebase -i` ?
kvdveer 5 hours ago [-]
I feel that grouping related change in commits can be challenging, as git really presents commits as grouping in time, not topic.
It is certainly possible to do topic-grouping in commits, but it requires significant effort to het that consistent on a team level.
tfrancisl 5 hours ago [-]
I concur. I cannot accept that we are so disconnected from what we're building that we can't go back and revise our commits or something else to make it make sense.
te_chris 18 minutes ago [-]
I’ve built this into a cli TUI. Passes the whole diff to Claude code with a schema and gets a structured narrative back out. Works really well for understanding.
Reconstituting messy things is exactly where LLMs can help.
tfrancisl 5 hours ago [-]
Why is this a service and not an open source project? It doesn't seem to do much other than organize your commits within a PR (could be run once on a dev machine and shipped in the code, then displayed separately) and builds a dashboard for PRs that's not too far off from what github already offers, but could also be represented with fairly small structured data and displayed separately.
- integrated AI review (uses codex and claude code defaults)
Stage (op product) navigation tour is nice UX, about a day worth of work in addition to the incoming code tour.
tfrancisl 2 hours ago [-]
this is for AI agent work though. That's cool, but not every team that wants better UX for complex work uses agents. Even if it "just works" for real scenarios, the marketing could be better.
ramoz 56 minutes ago [-]
Fair. There are users who simply just use the diff and integrated GitHub view/comment/approval-sync experience for local reviews of PRs. But it's _marketed_ as an integrated agent experience.
cpan22 4 hours ago [-]
Open source is something we're thinking about! We've just been focused on building for now but its definitely not off the table
tfrancisl 1 hours ago [-]
"Building" is always easier when you have a community that is ready and able to rout out bugs and suggest new features. Closed source makes that much less practical and appealing for most.
cpan22 57 minutes ago [-]
Totally get that, still something we're actively talking about!
Sort of related to that, we've been thinking a lot about the future of code review for OSS. Its clear with Cal.com going closed source that something needs to change. Would love to hear any thoughts you have
tfrancisl 49 minutes ago [-]
Cal.com going closed source was, without a doubt, shortsighted and unwise. I would recommend the blog post from the maintainers of Discourse on this.
pferde 2 hours ago [-]
Translation: we're hoping for an acqui-hire from some rich company, and will opensource this thing if it flops.
gracealwan 23 hours ago [-]
Totally different part of the reviewing experience, but I would love to see PR comments (or any revisions really) be automatically synced back to the context coding agents have about a codebase or engineer. There’s no reason nowadays for an engineer or a team of engineers to make the same code quality mistake twice. We manually maintain our agents.md with codebase conventions, etc, but it’d be great not to have to do that.
dean_stratakos 23 hours ago [-]
100%. A big part of code review in my mind is to automate away specific mistakes and anti-patterns across a team. I think there are a lot of interesting things to be done to merge the code writing and code reviewing cycles.
It keeps a repository with markdown files as the agent context, makes those available (via a simple search and summarise MCP) and when closing a merge request it checks whether the context needs updating based on the review comments. If it needs updating a PR is opened on the context repository with suggested changes/additions.
phyzix5761 5 hours ago [-]
This is a really cool idea but where's the moat? What's stopping someone from replicating the functionality?
cpan22 5 hours ago [-]
Thanks! I think we're really focused on making the overall review experience as guided and obvious as possible for the human. Chapters is a great start but we're coming up with more ideas on how we can make the process even easier
namanyayg 5 hours ago [-]
Looks amazing. I've been trying different stacking PR tools and Graphite and this looks to be the most human-centric so far. I'll have a shot at using this within our team soon. Congrats on the launch!
cpan22 4 hours ago [-]
Thank you! Let us know any ways we can make it better
lisayang888 2 hours ago [-]
Really like this idea. But at what point do you think its valuable to have this chapters breakdown versus splitting things up into multiple PRs?
dean_stratakos 1 hours ago [-]
Yeah it's getting easier to have agents just add "one more thing" to a PR, and I think there is still an aspect of human engineering judgement to know when to break up PRs versus trusting AI/tools to keep velocity high.
In the ideal world, each PR is as small and self-contained as possible but we've noticed people struggling to justify the extra overhead every time.
whywhywhywhy 6 hours ago [-]
The idea of a workplace where people can’t be bothered to read what the ai is coding but someone else is expected to read and understand if it’s good or slop just doesn’t really add up.
I personally see the value of code review but I promise you the most vocal vibe coders I work with don’t at all and really it feels like something that could be just automated to even me.
The age of someone gatekeeping the codebase and pushing their personal coding style foibles on the rest of the team via reviews doesn’t feels like something that will exist anymore if your ceo is big on vibe coding.
dean_stratakos 5 hours ago [-]
Agree that agents are definitely handling more and more of the coding side, and there's almost no doubt they will get better slop-wise.
In our view, even vibe coders should understand how the codebase works, and we think review is a natural place to pause and make sure you know what you and your coworkers are shipping. And we should have tools to reduce the mental load as much as possible.
Do you think there's a problem of cognitive debt among your coworkers who aren't reading the code or reviewing PRs?
SkyPuncher 4 hours ago [-]
Hmm. All of the examples simply describe what the code is doing. I need a tool that explains the intent and context behind a change.
lbreakjai 1 hours ago [-]
If you've worked from a plan, it's trivial. I've got a setup where the agent reads the implementation plan, then creates a commit history based on intent rather than location.
munksbeer 4 hours ago [-]
Exactly. "Why was this change made"? "What were the options"? "Why this is a good way of doing it"? "What are the subtle things I came across while making this change"?
cpan22 4 hours ago [-]
Yep that's something we're actively working on! would love to hear any perspectives on best ways to approach this
ryanjso 1 days ago [-]
I like the chapters thing, a lot of PRs I review should really be like 5 prs so its nice to have it auto split like that.
Do you see a world where it splits them up on the git level?
tasuki 6 hours ago [-]
> a lot of PRs I review should really be like 5 prs
Can't you push back on that? I feel like this tool is trying to fix misbehaved colleagues...
ryanjso 5 hours ago [-]
Yeah, but we're a small company and sometimes cut corners to move faster, so if a tool can solve this instead of potentially adding more friction to other engineers I'm all for it.
dean_stratakos 1 days ago [-]
Yeah that could be useful, especially with the increased popularity of stacked PRs
But I see it working together with chapters, not instead of bc it's still good to see the granularity within a PR
sscarduzio 22 hours ago [-]
We have the same problem, and I came up with this:
Basically it’s distilling knowledge from pr reviews back into Bugbot fine tuning and CLAUDE.md
So the automatic review catches more, and code assistant produces more aligned code.
cpan22 22 hours ago [-]
This is really cool and we definitely have this problem as well. I really like the flowchart deciding on where to put each learning. Will have to try it out!
Do you find that this list of learnings that end up BUGBOT.md or LESSONS.md ever gets too long? Or does it do a good job of deduplicating redundant learnings?
sscarduzio 14 hours ago [-]
Thanks! We have ~1000PRs/year. Seniors are way less than juniors and a lot of knowledge is transferred via pr messages.
The deduplication and generalisation steps really help, and the extra bugbot context ends up in just about 2000 tok.
Global LESSONS.md has less than 20 “pearls” with brief examples
cpan22 4 hours ago [-]
Nice! Will try it out
malcolmgreaves 4 hours ago [-]
Y’all are a bit nuts if you want 50% more per month than Claude Pro for this.
electrum 6 hours ago [-]
Does Stage work for PRs that have multiple commits? These could be considered "stacked diffs", but in the same PR.
dean_stratakos 6 hours ago [-]
Chapters are regenerated every time a new commit is pushed to a PR. Our thinking is that the chapters should serve as "auto stacked diffs" since they should follow a logical order.
Do you or your team use stacking in your workflows?
kylestlb 5 hours ago [-]
I assume Gitlab/Github will add these sort of features to their products within the next few months
dean_stratakos 5 hours ago [-]
It's possible, but at the same time it's been years and they haven't copied things like Graphite's dashboard or stacked PR interface yet. We have the advantage of speed :)
1 days ago [-]
sebakubisz 10 hours ago [-]
Can reviewers adjust the chapter splits manually if they disagree with how it grouped the PR, or are the chapters fixed once generated?
dean_stratakos 6 hours ago [-]
We don't support that currently, but would love to see examples where you disagree with the chapters so we can figure out the best interface
You can regenerate the chapters anytime, but it might lead to similar results as the first time
cpan22 5 hours ago [-]
We're also planning on adding functionality to support some sort of CHAPTERS.md file that lets you specify how you want things broken down!
sebakubisz 4 hours ago [-]
CHAPTERS.md sounds like a good idea for when the auto-grouping doesn't match the actual structure of the work. Looking forward to seeing it.
Thanks! Yeah we believe strongly that humans need to be in the code review loop to some extent
I think one thing we've seen from early users that surprised us is how chapters was quickly becoming the unit of review for them as opposed to files - and they've asked us to add functionality to mark chapters as viewed and comment on them as a whole
Another big surprise: now that agents are the ones writing most (if not all) the code right now, we've found that a lot of early users are using Stage to not only review others PRs but also their own PRs, before they have others review it
If AI is good enough to explain what the change is and call out what to focus on in the review, then why isn't AI good enough to just do the review itself?
I understand that the goal of this is to ensure there's still a human in the review cycle, but the problem I see is that suggestions will quickly turn into todo lists. Devs will read the summary, look at the what to review section, and stop reviewing code outside of things called out in the what to focus on section. If that's true, it means customers need to be able to trust that the AI has enough context to generate accurate summaries and suggestions. If the AI is able to generate accurate summaries and suggestions, then why can't we trust it to just do the review itself?
I'm not saying that to shit on the product, because I do get the logic behind it, but I think that's a question you should have a prepared answer for since I feel like I can't be the only one thinking that.
I think our perspective is that: software design has always had a subjective element to it. There's never been a "right" way to design a system, there are always trade offs that have to be made that depend on things like business context etc.
To that extent, most engineers probably still want to be part of that decision making process and not just let agents make all the high level decisions, especially if they're responsible for the code that ultimately gets merged
Your solution here seems to exclusively surface "what" changes, but it's impossible for me to know if it's right or not, unless I also see the "how" first and/or together with the change itself. So the same problem remains, except instead of reviewing in git/GitHub/gerrit + figure out the documents/resources that lays out the task itself, I still have to switch and confirm things between the two.
Currently on Stage we also generate a PR summary next to the chapters and that's where we want to do more "why" that pulls in context from Linear, etc.
And I know there's a lot of cool teams like Mesa and Entire working on embedding agent context into git history itself so that could an interesting area to explore as well
You cannot solve this problem by adding more AI on top. If lack of understanding is the problem, moving people even further away will only worsen the situation.
We don't think of Stage as moving people further away from code review, but rather using AI to guide human attention through the review process itself
AI guiding human attention means that humans aren't guiding human attention, which means less human understanding of their reviews.
So getting assistance in the review, in making the decisions and giving me more clarity feels interesting.
Maybe its people like me, who became involved into coding after the LLMs who might be your niche.
One thing I dont understand, the UI/UX? Is this visible only on git itself? Or can I get it working in Codex?
We've wondered about what the review experience should look like for newly technical or non-technical people now that they are increasingly putting up PRs themselves. These people will be less opinionated about certain technical decisions in general so maybe the future looks like review processes very personalized to your experience level and your background. Definitely a lot to think about
Right now the chapters UI is only available on our website but we're exploring possible integrations and/or a desktop app
Isn't that what commits are for? I see no reason for adding this as an after-thought. If the committers (whether human or LLM) are well-behaved, this info is already available in the PR.
We thought git wasn't the right level of abstraction and decided to tackle things at the PR level instead. Curious to hear your experiences!
Sure, it is. But it's worth it, not just for code review, but for a myriad other things: bisect, blame, log, etc.
Your tool makes one thing (the code review) easier, while decreasing people's motivation to make well-mannered commits, thus making everything else (bisect etc) worse.
I'm sure it's net positive in some cases, and I think it's net negative in other cases.
Have you heard about `rebase -i` ?
It is certainly possible to do topic-grouping in commits, but it requires significant effort to het that consistent on a team level.
Reconstituting messy things is exactly where LLMs can help.
and a code tour feature about to ship: https://x.com/backnotprop/status/2043759492744270027/video/1
- integrated comment feedback for agents
- inline chat
- integrated AI review (uses codex and claude code defaults)
Stage (op product) navigation tour is nice UX, about a day worth of work in addition to the incoming code tour.
Sort of related to that, we've been thinking a lot about the future of code review for OSS. Its clear with Cal.com going closed source that something needs to change. Would love to hear any thoughts you have
It keeps a repository with markdown files as the agent context, makes those available (via a simple search and summarise MCP) and when closing a merge request it checks whether the context needs updating based on the review comments. If it needs updating a PR is opened on the context repository with suggested changes/additions.
In the ideal world, each PR is as small and self-contained as possible but we've noticed people struggling to justify the extra overhead every time.
I personally see the value of code review but I promise you the most vocal vibe coders I work with don’t at all and really it feels like something that could be just automated to even me.
The age of someone gatekeeping the codebase and pushing their personal coding style foibles on the rest of the team via reviews doesn’t feels like something that will exist anymore if your ceo is big on vibe coding.
In our view, even vibe coders should understand how the codebase works, and we think review is a natural place to pause and make sure you know what you and your coworkers are shipping. And we should have tools to reduce the mental load as much as possible.
Do you think there's a problem of cognitive debt among your coworkers who aren't reading the code or reviewing PRs?
Do you see a world where it splits them up on the git level?
Can't you push back on that? I feel like this tool is trying to fix misbehaved colleagues...
But I see it working together with chapters, not instead of bc it's still good to see the granularity within a PR
https://sscarduzio.github.io/pr-war-stories/
Basically it’s distilling knowledge from pr reviews back into Bugbot fine tuning and CLAUDE.md
So the automatic review catches more, and code assistant produces more aligned code.
Do you find that this list of learnings that end up BUGBOT.md or LESSONS.md ever gets too long? Or does it do a good job of deduplicating redundant learnings?
The deduplication and generalisation steps really help, and the extra bugbot context ends up in just about 2000 tok.
Global LESSONS.md has less than 20 “pearls” with brief examples
Do you or your team use stacking in your workflows?
You can regenerate the chapters anytime, but it might lead to similar results as the first time
I think one thing we've seen from early users that surprised us is how chapters was quickly becoming the unit of review for them as opposed to files - and they've asked us to add functionality to mark chapters as viewed and comment on them as a whole
Another big surprise: now that agents are the ones writing most (if not all) the code right now, we've found that a lot of early users are using Stage to not only review others PRs but also their own PRs, before they have others review it