Skip to content

Conversation

@devmotion
Copy link
Contributor

Very much WIP, and I might have to start over again. Fixes #1210 (comment) by not (mis)using objective functions as states. Instead, this PR proposes to add f_x and g_x fields to the optimization states (where needed), similar to already existing x_previous, f_x_previous and g_x_previous fields.

@devmotion devmotion force-pushed the dmw/do_not_misuse_objective_as_state branch 2 times, most recently from 935c749 to 93e8f31 Compare November 26, 2025 12:27
end

if g_calls(d) > 0 && !all(isfinite, gradient(d))
if hasproperty(state, :g_x) && !all(isfinite, state.g_x)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could in principle add methods to isfinite_gradient and isfinite_hessian that you introduced for the objective? I suppose they are sufficiently similar in nature to share the function?

Copy link
Member

@pkofod pkofod Nov 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if it's only used once of course... then it's maybe overkill :) I suppose in termination code as well..

@devmotion devmotion force-pushed the dmw/do_not_misuse_objective_as_state branch 6 times, most recently from c4d1eec to d2071a1 Compare November 27, 2025 22:20
@devmotion devmotion force-pushed the dmw/do_not_misuse_objective_as_state branch from 8c40315 to 60a5fef Compare November 28, 2025 10:58
@github-actions
Copy link
Contributor

github-actions bot commented Nov 28, 2025

Benchmark Results (Julia vlts)

Time benchmarks
master 6182632... master / 6182632...
multivariate/solvers/first_order/AdaMax 0.543 ± 0.0098 ms 0.543 ± 0.0094 ms 1 ± 0.025
multivariate/solvers/first_order/Adam 0.542 ± 0.0096 ms 0.541 ± 0.0094 ms 1 ± 0.025
multivariate/solvers/first_order/BFGS 0.262 ± 0.0088 ms 0.261 ± 0.0086 ms 1 ± 0.047
multivariate/solvers/first_order/ConjugateGradient 0.175 ± 0.0033 ms 0.181 ± 0.0032 ms 0.965 ± 0.025
multivariate/solvers/first_order/GradientDescent 1.55 ± 0.012 ms 1.54 ± 0.012 ms 1 ± 0.011
multivariate/solvers/first_order/LBFGS 0.234 ± 0.0075 ms 0.232 ± 0.008 ms 1.01 ± 0.047
multivariate/solvers/first_order/MomentumGradientDescent 2.17 ± 0.015 ms 2.17 ± 0.018 ms 0.997 ± 0.011
multivariate/solvers/first_order/NGMRES 0.434 ± 0.012 ms 0.429 ± 0.012 ms 1.01 ± 0.04
time_to_load 0.439 ± 0.0052 s 0.443 ± 0.0046 s 0.991 ± 0.016
Memory benchmarks
master 6182632... master / 6182632...
multivariate/solvers/first_order/AdaMax 0.339 k allocs: 7.09 kB 0.34 k allocs: 7.19 kB 0.987
multivariate/solvers/first_order/Adam 0.339 k allocs: 7.09 kB 0.34 k allocs: 7.19 kB 0.987
multivariate/solvers/first_order/BFGS 0.359 k allocs: 15.5 kB 0.336 k allocs: 15 kB 1.03
multivariate/solvers/first_order/ConjugateGradient 0.361 k allocs: 14.1 kB 0.338 k allocs: 13.6 kB 1.04
multivariate/solvers/first_order/GradientDescent 2.08 k allocs: 0.0758 MB 1.89 k allocs: 0.0713 MB 1.06
multivariate/solvers/first_order/LBFGS 0.34 k allocs: 14.7 kB 0.317 k allocs: 14.2 kB 1.03
multivariate/solvers/first_order/MomentumGradientDescent 2.44 k allocs: 0.0815 MB 2.24 k allocs: 0.077 MB 1.06
multivariate/solvers/first_order/NGMRES 1.56 k allocs: 0.117 MB 1.41 k allocs: 0.112 MB 1.05
time_to_load 0.153 k allocs: 14.5 kB 0.153 k allocs: 14.5 kB 1

A plot of the benchmark results has been uploaded as an artifact at .

@devmotion devmotion force-pushed the dmw/do_not_misuse_objective_as_state branch 5 times, most recently from 938f2ec to 95ca2d8 Compare November 30, 2025 21:54
@codecov
Copy link

codecov bot commented Nov 30, 2025

Codecov Report

❌ Patch coverage is 87.84067% with 58 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.77%. Comparing base (9290b55) to head (6182632).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
src/multivariate/solvers/first_order/adam.jl 59.37% 13 Missing ⚠️
src/multivariate/solvers/first_order/adamax.jl 59.37% 13 Missing ⚠️
...tivariate/solvers/constrained/ipnewton/interior.jl 63.63% 12 Missing ⚠️
src/multivariate/solvers/constrained/samin.jl 80.00% 6 Missing ⚠️
...ariate/solvers/second_order/newton_trust_region.jl 79.16% 5 Missing ⚠️
src/multivariate/solvers/first_order/bfgs.jl 84.00% 4 Missing ⚠️
src/multivariate/solvers/constrained/fminbox.jl 97.43% 1 Missing ⚠️
...tivariate/solvers/constrained/ipnewton/ipnewton.jl 95.65% 1 Missing ⚠️
src/multivariate/solvers/second_order/newton.jl 91.66% 1 Missing ⚠️
src/utilities/assess_convergence.jl 95.23% 1 Missing ⚠️
... and 1 more
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1212      +/-   ##
==========================================
- Coverage   87.32%   86.77%   -0.55%     
==========================================
  Files          45       45              
  Lines        3558     3539      -19     
==========================================
- Hits         3107     3071      -36     
- Misses        451      468      +17     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@devmotion devmotion force-pushed the dmw/do_not_misuse_objective_as_state branch from 95ca2d8 to 92ae7aa Compare November 30, 2025 22:09
@devmotion devmotion marked this pull request as ready for review November 30, 2025 22:33
@devmotion devmotion changed the title [WIP] Do not (mis)use objective as state Do not (mis)use objective as state Nov 30, 2025
@devmotion devmotion force-pushed the dmw/do_not_misuse_objective_as_state branch from 92ae7aa to 6182632 Compare November 30, 2025 22:47
@pkofod
Copy link
Member

pkofod commented Dec 1, 2025

I suppose the artifacts need to be uniquely named

@devmotion
Copy link
Contributor Author

Indeed, I already opened a PR on Friday: MilesCranmer/AirspeedVelocity.jl#120

@pkofod pkofod merged commit a466ee4 into JuliaNLSolvers:master Dec 1, 2025
13 of 14 checks passed
@devmotion devmotion deleted the dmw/do_not_misuse_objective_as_state branch December 1, 2025 12:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants