Skip to content

[Storage] Replacing scaling logs to WebJobs extension methods #53923

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
+19 −12
Open

[Storage] Replacing scaling logs to WebJobs extension methods #53923

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions NuGet.Config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
<clear />
<!-- Do not add any additional feeds if new packages are needed they need to come from our azure-sdk-for-net DevOps feed which has an upstream set to nuget.org -->
<add key="azure-sdk-for-net" value="https://pkgs.dev.azure.com/azure-sdk/public/_packaging/azure-sdk-for-net/nuget/v3/index.json" />
<add key="local" value="Q:\nuget" />
Copy link

Copilot AI Nov 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A local package source Q:\nuget has been added to the NuGet configuration. This appears to be a development-specific path that should not be committed to the repository.

Please remove this line before merging, as it will not work for other developers and could cause build issues in CI/CD pipelines.

Suggested change
<add key="local" value="Q:\nuget" />

Copilot uses AI. Check for mistakes.
</packageSources>
<disabledPackageSources>
<clear />
Expand Down
4 changes: 2 additions & 2 deletions eng/Packages.Data.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -249,8 +249,8 @@
<PackageReference Update="Microsoft.Azure.SignalR.Management" Version="1.29.0" />
<PackageReference Update="Microsoft.Azure.SignalR.Protocols" Version="1.29.0" />
<PackageReference Update="Microsoft.Azure.SignalR.Serverless.Protocols" Version="1.10.0" />
<PackageReference Update="Microsoft.Azure.WebJobs" Version="3.0.41" />
<PackageReference Update="Microsoft.Azure.WebJobs.Sources" Version="3.0.41" PrivateAssets="All"/>
<PackageReference Update="Microsoft.Azure.WebJobs" Version="3.0.42-dev" />
<PackageReference Update="Microsoft.Azure.WebJobs.Sources" Version="3.0.42-dev" PrivateAssets="All"/>
Comment on lines +252 to +253
Copy link

Copilot AI Nov 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The package versions have been updated to 3.0.42-dev, which appears to be a development/pre-release version. Please verify:

  1. This version exists in the package source (likely the local source added in NuGet.Config)
  2. The final PR should use a stable version or a properly published pre-release version
  3. If this is intentional for testing the new extension methods, ensure it's documented and that the package will be available when this PR is merged

The -dev suffix suggests this might be a locally built package, which would require other developers to also build this package locally.

Suggested change
<PackageReference Update="Microsoft.Azure.WebJobs" Version="3.0.42-dev" />
<PackageReference Update="Microsoft.Azure.WebJobs.Sources" Version="3.0.42-dev" PrivateAssets="All"/>
<PackageReference Update="Microsoft.Azure.WebJobs" Version="3.0.42" />
<PackageReference Update="Microsoft.Azure.WebJobs.Sources" Version="3.0.42" PrivateAssets="All"/>

Copilot uses AI. Check for mistakes.
<PackageReference Update="Microsoft.Azure.WebJobs.Extensions.Rpc" Version="3.0.41" />
<PackageReference Update="Microsoft.Azure.WebJobs.Host.Storage" Version="5.0.1" />
<PackageReference Update="Microsoft.Spatial" Version="7.5.3" />
Expand Down
15 changes: 10 additions & 5 deletions ...crosoft.Azure.WebJobs.Extensions.Storage.Common/src/Shared/Queues/QueueMetricsProvider.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
using Azure;
using Azure.Storage.Queues;
using Azure.Storage.Queues.Models;
using Microsoft.Azure.WebJobs.Host.Scale;
using Microsoft.Extensions.Logging;

namespace Microsoft.Azure.WebJobs.Extensions.Storage.Common.Listeners
Expand All @@ -15,16 +16,19 @@ namespace Microsoft.Azure.WebJobs.Extensions.Storage.Common.Listeners
/// </summary>
internal class QueueMetricsProvider
{
private readonly string _functionId;
private readonly QueueClient _queue;
private readonly ILogger _logger;

/// <summary>
/// Instantiates a QueueMetricsProvider.
/// </summary>
/// <param name="functionId">The function id to make scale decisions for.</param>
/// <param name="queue">The QueueClient to use for metrics polling.</param>
/// <param name="loggerFactory">Used to create an ILogger instance.</param>
public QueueMetricsProvider(QueueClient queue, ILoggerFactory loggerFactory)
public QueueMetricsProvider(string functionId, QueueClient queue, ILoggerFactory loggerFactory)
{
_functionId = functionId;
_queue = queue;
_logger = loggerFactory.CreateLogger<QueueMetricsProvider>();
}
Expand All @@ -49,12 +53,12 @@ public async Task<int> GetQueueLengthAsync()
// ignore transient errors, and return default metrics
// E.g. if the queue doesn't exist, we'll return a zero queue length
// and scale in
_logger.LogWarning($"Error querying for queue scale status: {ex.ToString()}");
_logger.LogFunctionScaleWarning("Error querying for queue scale status", _functionId, ex);
}
}
catch (Exception ex)
{
_logger.LogWarning($"Fatal error querying for queue scale status: {ex.ToString()}");
_logger.LogFunctionScaleWarning("Fatal error querying for queue scale status", _functionId, ex);
}

return 0;
Expand Down Expand Up @@ -101,12 +105,13 @@ public async Task<QueueTriggerMetrics> GetMetricsAsync()
// ignore transient errors, and return default metrics
// E.g. if the queue doesn't exist, we'll return a zero queue length
// and scale in
_logger.LogWarning($"Error querying for queue scale status: {ex.ToString()}");

Copy link
Member

@mathewc mathewc Nov 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: looks like you included an extraneous space here?

_logger.LogFunctionScaleWarning("Error querying for queue scale status", _functionId, ex);
}
}
catch (Exception ex)
{
_logger.LogWarning($"Fatal error querying for queue scale status: {ex.ToString()}");
_logger.LogFunctionScaleWarning("Fatal error querying for queue scale status", _functionId, ex);
}

return new QueueTriggerMetrics
Expand Down
2 changes: 1 addition & 1 deletion .../Microsoft.Azure.WebJobs.Extensions.Storage.Common/src/Shared/Queues/QueueScaleMonitor.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ public QueueScaleMonitor(string functionId, QueueClient queue, ILoggerFactory lo
_queue = queue;
_logger = loggerFactory.CreateLogger<QueueListener>();
_scaleMonitorDescriptor = new ScaleMonitorDescriptor($"{functionId}-QueueTrigger-{_queue.Name}".ToLower(CultureInfo.InvariantCulture), functionId);
_queueMetricsProvider = new QueueMetricsProvider(queue, loggerFactory);
_queueMetricsProvider = new QueueMetricsProvider(functionId, queue, loggerFactory);
}

/// <summary>
Expand Down
5 changes: 3 additions & 2 deletions .../Microsoft.Azure.WebJobs.Extensions.Storage.Common/src/Shared/Queues/QueueTargetScaler.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ internal QueueTargetScaler(string functionId, QueueClient queueClient, QueuesOpt
{
_functionId = functionId;
_queueName = queueClient.Name;
_queueMetricsProvider = new QueueMetricsProvider(queueClient, loggerFactory);
_queueMetricsProvider = new QueueMetricsProvider(_functionId, queueClient, loggerFactory);
_targetScalerDescriptor = new TargetScalerDescriptor(functionId);
_options = options;
_logger = loggerFactory.CreateLogger<QueueTargetScaler>();
Expand Down Expand Up @@ -64,7 +64,8 @@ internal TargetScalerResult GetScaleResultInternal(TargetScalerContext context,

int targetWorkerCount = (int)Math.Ceiling(queueLength / (decimal)concurrency);

_logger.LogInformation($"Target worker count for function '{_functionId}' is '{targetWorkerCount}' (QueueName='{_queueName}', QueueLength ='{queueLength}', Concurrency='{concurrency}').");
string details = $"Target worker count for function '{_functionId}' is '{targetWorkerCount}' (QueueName='{_queueName}', QueueLength ='{queueLength}', Concurrency='{concurrency}').";
_logger.LogFunctionScaleVote(_functionId, targetWorkerCount, queueLength, concurrency, details);
return new TargetScalerResult
{
TargetWorkerCount = targetWorkerCount
Expand Down
4 changes: 2 additions & 2 deletions ...rage/Microsoft.Azure.WebJobs.Extensions.Storage.Queues/tests/QueueMetricsProviderTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ public void SetUp()
_loggerFactory.AddProvider(_loggerProvider);
_mockQueue = new Mock<QueueClient>(new Uri("https://test.queue.core.windows.net/testqueue"), new QueueClientOptions());
_mockQueue.Setup(x => x.Name).Returns("testqueue");
_metricsProvider = new QueueMetricsProvider(_mockQueue.Object, _loggerFactory);
_metricsProvider = new QueueMetricsProvider("testfunction", _mockQueue.Object, _loggerFactory);
}

[OneTimeSetUp]
Expand All @@ -54,7 +54,7 @@ public void OneTimeTearDown()
[Test]
public async Task GetMetrics_ReturnsExpectedResult()
{
QueueMetricsProvider _provider = new QueueMetricsProvider(Fixture.Queue, _loggerFactory);
QueueMetricsProvider _provider = new QueueMetricsProvider("testfunction", Fixture.Queue, _loggerFactory);
var metrics = await _provider.GetMetricsAsync();

Assert.AreEqual(0, metrics.QueueLength);
Expand Down
Loading