🚀 Speculative agent autoscaling

[Shweta-Deshpande]

Rollback Plan

If a change needs to be reverted, we will publish an updated version of the library.

Changes to Security Controls

Are there any changes to security controls (access controls, encryption, logging) in this pull request? If so, explain.

Description

Adding support for scaling agents when plan-only runs are queued. This will run them parallelly and improve performance for workspaces with large speculative runs queued for each pull request.
Currently, the new algorithm that calculates the agents required for executing pending runs, counts one agent per workspace. This does not scale for speculative runs where we don't need runs to wait in a queue and run them on agents parallelly instead. This change will increment the required agents count when there is run of type "plan-only" which will resolve the issue.

Usage Example

References

Community Note

• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.

[arybolovlev]

All in all, looks good, thank you! 👍🏻 I’ve left a few cosmetic, logging-related comments.

🪵 Since this change introduces a new way of calculating pending runs, it needs to be clearly documented in the changelog. This entry is currently missing. The PR description can provide additional context, so anyone who wants to understand the change in more detail can refer to it.

Mock tests are something we have missed. However, there are a few issues with them (I might be missing something, so please correct me):

• There is no new make target for running them separately.

• They currently run together with the E2E tests and fail.

• As mentioned above, the mocks package from go-tfe is imported, and a new folder/file has been added, but it is not used.

We can either postpone these as part of a larger testing effort or fix them properly now.

[Shweta-Deshpande]

All in all, looks good, thank you! 👍🏻 I’ve left a few cosmetic, logging-related comments.

🪵 Since this change introduces a new way of calculating pending runs, it needs to be clearly documented in the changelog. This entry is currently missing. The PR description can provide additional context, so anyone who wants to understand the change in more detail can refer to it.

Mock tests are something we have missed. However, there are a few issues with them (I might be missing something, so please correct me):

• There is no new make target for running them separately.
• They currently run together with the E2E tests and fail.
• As mentioned above, the mocks package from go-tfe is imported, and a new folder/file has been added, but it is not used.

We can either postpone these as part of a larger testing effort or fix them properly now.

I fixed the failure in the test. It strangely worked well when I created the test first.