Time for a new Batchspawner release

[mbmilligan]

I think we have enough changes queued up to merit a new release in the near future.

Consider this space a placeholder - we will fill it in with a list of issues we want to resolve/PRs we want to pull for this release.

I will go through the list of these items and choose some, but in the meantime if folks want to nominate items for the release please leave them in the comments on this issue. Thanks!

@rkdarst @cmd-ntrf @willingc

[rkdarst]

My thoughts:

Pull (of these my integration branch has #99, #97, #96, #94, #106/#107 and it seems to work):

• Travis: Fix tests (jsonschema!=3.0.0a1) and update for jupyterhub 9.0.1/python3.7 #105 (fix travis tests)
• Update exec_prefix handling: template this separate from batch command. #94
• Add prologue and epilogue options to other spawners #96
• Miscellaneous minor tune ups, shouldn't effect functioning  #100
• run_command: Increase debugging output in case of error #106 (assuming tests pass)
• Add reservation option for Slurm #91 (maybe, not currently used, and do we need a traitlet for every option when req_options is quite general?)
• Select singleserver port number on remote host #58 (if updated)
• Keepvars extra #99 (only important if someone thinks about Consider security of environment #82, but it should be ready to go so if OK then why not now?)
• run_command: Fix possible deadlock #107 (if tests pass, probably won't in which case don't pull)
• Pass the environment dict to all commands (query and cancel) #111 (environment to queue/cancel commands)

To do:

• finalize [WIP] Add SPAWNERS.md, a list of specific info on spawners #97 (assignee: @rkdarst)
• port range restricting port range for spawned jupyterhub singleuser servers #102 (I can make new patch if needed) or Select random port from a range (new implementation) #114 (different implementation, fails with JH0.7.2)
• Adding PBSSpawner #109 (PBS spawner, if finalized)

we can possibly close:

• how to include a "extra launch script" #98
• Any way to not have public notebook servers? #31

[mbmilligan]

Thanks for your work going through all those! With the caveat that we can always change our mind if some item becomes problematic, I'm okay using this as a starting point.

[rkdarst]

#112 has been created which integrates everything needed for the release. Just like the last release, waiting for PRs one at a time seemed to not be very feasible.

I'd say mainly we are waiting for confirmation on #109 and then any other changes from maintainers.

Please comment on there, and anyone can feel free to make pushes or PRs to that branch to support the release.

[mbmilligan]

Ok, now that the big integration branch is merged, we're almost there. Thanks @rkdarst for doing most of the actual work!

Is it correct that #109 did not go in with the dev branch merge? If not, it looks like it's ready to go in.

The remaining issues appear to all relate to port number setting. Where do we stand on #102 vs #114 - and have they been tested to not conflict with #58 ?

[rkdarst]

Correct, #109 wasn't there. But it reports green now so should be good to go.

For the port number things... #114 fails on JH 0.7.2, I looked and superficially it seems like it should still work, but I guess somewhere else it doesn't use the returned port. Either someone needs to dig deeper, or we can skip the test on 0.7.2.

And it will certainly require some merging with #58, both code level and JH level. For example, currently info on port selection isn't passed to the singleuser server at all, so it's (port_range function) or (remote port selection) but not both. Someone with deeper knowledge than me should evaluate #58 and ensure everything will go smoothly, in particular there are no weird problems with asyncness or whatever. Maybe I can get to this sometime, but I'm not the best one here.

[cmd-ntrf]

Regarding #58, I have updated the PR to allow port selection, fixed the tests and have the code up to date with the master branch. I have also put some explanations on why some tests fail in the PR comments.

@rkdarst Let me know if I can anything to ease the merge of PR #58.

[guillaumeeb]

High everyone, is the release planned anytime soon? Let me know if something is needed in #109.

I'd like to have a clean release to make an "official" installation of Jupyterhub in my organization.

[rkdarst]

I'd like to do the release, but I can't myself. It would be good if someone could comment on other PRs.

I occasionally make my own integration branches when stuff is too slow, but that's more of a workaround when I need to deploy. I imagine I will get to upgrading our batchspawner within a few weeks, so I'll probably make a big push then.

Is there any way that the community can help with batchspawner more?

[guillaumeeb]

Will be happy to help more too.

[mbmilligan]

Merged #109. Will dig into #58 and #114 in the next day or two and see if we can come to a good resolution.

[mbmilligan]

Merged @willingc tweaked version of PR #115 (which became #124 ) and now tests are working again, hooray. #58 appears to be in a good state now, so I've merged that too. It is available in the main branch so we can decide if #114 will need to work around it.

@rkdarst and others, are there other PRs that need merging ahead of cutting a new release?

[mbmilligan]

Just a note: now that we are getting a little bit of testing on the current state of master, I think we need to sort out issues #126 and #127 before making a release. The changes do need documentation, but also I think users shouldn't have to add an import to their config file unless they are actually using the new feature.

[cmd-ntrf]

I agree, having users import batchspawner.api was never part of the plan. All of my tests were with batchspawner directly, which explain why I missed potential issue with ProfileSpawner.