Skip to content

Conversation

@RolandMinrui
Copy link
Contributor

@RolandMinrui RolandMinrui commented Jul 17, 2025

This pr is intended to:

  • Make the feedback of runner to multi-turn conversation.
  • Add a "role" or "status" description for each of agents. This should includes what does the pipeline do, what's your role in the pipeline, what is the percentage of runtime used and what is the loops used, etc.
  • Add change summary to runner.
  • Add previous loops information to runner.

@RolandMinrui RolandMinrui marked this pull request as draft July 17, 2025 06:18
queried_knowledge = self.rag.query(evo, self.evolving_trace)

# 2.5 Brief evolving history
evolving_history = (
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we defer therender it to last step

if self.with_feedback:
es.feedback = (
eva if isinstance(eva, Feedback) else eva.evaluate(evo, queried_knowledge=queried_knowledge)
eva if isinstance(eva, Feedback) else eva.evaluate(evo, evolving_history=evolving_history)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we have to remove queried_knowledge=queried_knowledge?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not used in current runner

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

But it is used in other evaluators...

evolvable_subjects: EvolvableSubjects
queried_knowledge: QueriedKnowledge | None = None
feedback: Feedback | None = None
code_change_summary: str | None = None # TODO: minrui
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this place is a good place to add summary,
Let's have the discussion later.

implementation: FBWorkspace,
gt_implementation: FBWorkspace,
queried_knowledge: QueriedKnowledge = None,
evolving_history: tuple = None,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

eolving_trace

{
"code": "The Python code as a string."
"code": "The refined Python code as a string."
"code_change_summary": "The structured summary to briefly summarize the changes made to the original code base in two to three sentences."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remember change

@RolandMinrui RolandMinrui requested a review from peteryang1 July 23, 2025 06:58
@RolandMinrui RolandMinrui marked this pull request as ready for review July 23, 2025 12:49
@RolandMinrui RolandMinrui force-pushed the minrui/add_runner_history branch from fd27cb7 to 1064b68 Compare July 25, 2025 03:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants