claude and CI - interactions

[niphlod]

Summarize Command's Functionality

Yeah, it's middle between a "bug" and a "feature request" but anyway ...
We have two major forces clashing ATM that may need more organized thoughts in order to be solved once and for all.
Clearly trying to address the issue in our spare time or asking AI isn't solving the issue.

Force number 1
claude is addictive. It works creating feature branches, committing multiple times, and being an AI needs multiple iterations to get to a proper result

Force number 2
AI is great. Having unit/integration tests coupled with AI makes sure we don't inadvertently break things. And it's the best way to let AI unravel til tests pass.
But, we recently switched from paid plan on appveyor to a free one, and we're suffering from long builds time.

Clearly, long CI builds make the whole "claude let's fix many little things" less convenient.

We tried a few variations of appveyor.yml, but we're always missing something.

If we draw it on a napkin what we're doing, though, I think it makes more clear that we need to rule things on the napkin before trying to fix either claude or appveyor.

The usual "flow"/"github flow" for a contributor:

• someone opens an issue/needs to fix something
• a contributor chooses the "issue" to work on
• a feature branch is spawned LOCALLY. Work on the code with as many commit as needed is done LOCALLY, till you have something that stands the chance of
  • being a completely working solution
  • is at least clearly readable
  • being reviewed by peers
• nothing of the "temp work"/"iterations"/"trials"/"variations" lands on the GH repo, only the final "solution"
• the introduction of "agents" doesn't break the flow. What changes is the agent committing locally rather than the human, but GH repo stays untouched the same way
• once code is ready, you push the code to your forked repo (or, in case of @potatoqualitee , in datapat/dbatools directly)
• you then issue a PR against development
• in case of @potatoqualitee , you open a PR from the feature branch to development

So, if you're like "me", an external contributor, dataplat/dbatools Github sees my code ONCE, when I issue the PR.
If you're like @potatoqualitee , dataplat/dbatools Github sees your code WHEN you push the branch plus WHEN you open the PR.
Appveyor is linked closely to Github activity: we can try and filter out things but as anticipated, it's better to have a napkin-draw of what is happening to figure out the best solution.

The problem here is that "democratizing" claude as a bot running on Github (which in by itself, is a good thing and I think it also highlights, publicly, how far can an AI go (winkwink, claude sponsorship)) has a side-effect.
all work to reach a situation where code works and implements what's needed, that was previously "hidden away" working only locally, is now instead landing on Github, commit by commit .

It's like forcing each external contributor to just work on dataplat/dbatools, without forks, everyone in this repo, plus autocommitting and pushing each 5 minutes.

So "the current flow" is basically 50x activity on Github, which then triggers 50x builds on Appveyor. It's not that "now it's not working" vs "we never had this issue".
It was happening even before, but activity on Github landed on "already-polished" code, at the end of the timeline of working on a specific bug/feature.

Hoping that it's now clear the "framing" of the general problem, looking at the last few weeks, what is happening (and I'm assuming it'll continue to happen) is....

1. you see an issue
2. you figure out claude can help (or, in case of @potatoqualitee, you challenge claude just for the sake of knowing what'll do :-P )
3. you mention claude in the issue
4. it replies with its reasoning
5. you optionally try and chat about it a little more
6. once satisfied, you tell it to prepare a PR
7. either within the chat, or when you tell it to prepare a PR, claude
   • spawns a new branch (claude/issue-.*) ON Github
   • starts adding commits there
8. every further interactions (a-la "claude please fix it differently, or better") creates more commits on claude-issue.*
9. you click on the PR "link" predisposed by claude
10. a proper PR is born
11. tests may fail miserably
12. you either fix code directly or you make claude fix its issues
13. you wait for tests to pass
14. you ask for a review from a proper human
15. you can finally merge to development

Now, I think we wanna avoid appveyor triggering builds at each commit.
So, basically, we wanna skip builds till 10.
Recent changes to the test runner should assure that on 10., a build is triggered but tests do run only on the function(s) changed within the PR.
I think we still need to run CI from 10. to 13. And just hope it doesn't involve 15 separate commits. But IF they're frequent, previous build on the same PR just gets canceled and a new one starts... so it should be "fine", I guess.

Are we all onboard with the aforementioned analysis (mostly @potatoqualitee but, anyone, really) ?

If yes, "process" is there and we can figure out the best solution and/or ask for directions.

Is there a command that is similiar or close to what you are looking for?

No

Technical Details

No response

[potatoqualitee]

Yes, waiting until #10 would be perfect. But if that's not possible then we can just remove all filters since you got the others down to such reasonable runtimes.

[niphlod]

is 7. happening the way I described ? i.e. claude starts altering code in its own branch even if you don't tell it explicitely to issue a pr ?

[potatoqualitee]

yes but only on github dot com. when you do it locally you have 100% control like usual