Skip to content

Commit 1a82fa4

Browse files
committed
enh(policy): add Gen AI policy to review guide
1 parent 136569f commit 1a82fa4

File tree

4 files changed

+77
-31
lines changed

4 files changed

+77
-31
lines changed

appendices/editor-in-chief-checks.md

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,12 +9,12 @@ to work on them before the review process begins.
99
Please check our [Python packaging guide](https://www.pyopensci.org/python-package-guide) for more information on the elements
1010
below.
1111

12-
- [ ] **Installation** The package can be installed from a community repository such as PyPI (preferred), and/or a community channel on conda (e.g., conda-forge, bioconda).
12+
- [ ] **Installation** The package can be installed from a community repository such as PyPI (preferred), and/or a community channel on conda (for example, conda-forge, bioconda).
1313
- [ ] The package imports properly into a standard Python environment `import package`.
1414
- [ ] **Fit** The package meets criteria for [fit](https://www.pyopensci.org/software-peer-review/about/package-scope.html#what-types-of-packages-does-pyopensci-review) and [overlap](https://www.pyopensci.org/software-peer-review/about/package-scope.html#package-overlap).
1515
- [ ] **Documentation** The package has sufficient online documentation to allow us to evaluate the package's function and scope *without installing the package*. This includes:
1616
- [ ] User-facing documentation that overviews how to install and start using the package.
17-
- [ ] Short quickstart tutorials that help a user understand how to use the package and what it can do for them.
17+
- [ ] Short, quickstart tutorials that help a user understand how to use the package and what it can do for them.
1818
- [ ] API documentation (documentation for your code's functions, classes, methods, and attributes): this includes clearly written docstrings with variables defined using a standard docstring format.
1919
- [ ] Core GitHub repository Files
2020
- [ ] **README** The package has a `README.md` file with a clear explanation of what the package does, instructions on how to install it, and a link to development instructions.
@@ -28,13 +28,14 @@ NOTE: We prefer that you have development instructions in your documentation too
2828
- [ ] **Package overlap** The package doesn't entirely overlap with the functionality of other packages that have already been submitted to pyOpenSci.
2929
- [ ] **Archive** (JOSS only, may be post-review): The repository DOI resolves correctly.
3030
- [ ] **Version** (JOSS only, may be post-review): Does the release version given match the GitHub release (v1.0.0)?
31+
- [ ] The package authors have disclosed the use of Generative AI tools in the development and/or maintenance of their package.
3132

3233
**Optional:** Let projects know that it's a great idea for projects to have a .github repository for the project organization where they can host a commonly used LICENSE, Code of Conduct, and even a YAML file with label definitions. These items will then be automatically applied to every repository in the organization to ensure consistency (but can be customized within repos too). The [SunPy project](https://github.com/sunpy/.github/) has a great example of this.
3334

3435
---
3536
- [ ] [Initial onboarding survey was filled out ](https://forms.gle/F9mou7S3jhe8DMJ16)
3637
We appreciate each maintainer of the package filling out this survey individually. :raised_hands:
37-
Thank you, authors, in advance for setting aside five to ten minutes to do this. It truly helps our organization. :raised_hands:
38+
Thank you, authors, in advance, xwfor setting aside five to ten minutes to do this. It truly helps our organization. :raised_hands:
3839
---
3940

4041
*******

appendices/gen-ai-checklist.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
```markdown
2+
- [ ] Some parts of the package were created using LLMs in some way.
3+
* Please check any of the boxes below to clarify which parts of the package were impacted by LLM use
4+
- [ ] LLMs were used to develop code
5+
- [ ] LLMs were used to develop documentation
6+
- [ ] LLMs were used to develop tests
7+
- [ ] LLMs were used to develop infrastructure (CI, automation)
8+
9+
- [ ] I acknowledge that a human (at least one member of our maintainer team) has reviewed, edited, and modified all LLM-generated code and text submitted in this review to ensure it’s not biased (if text) and the code runs (if code).
10+
```

how-to/author-guide.md

Lines changed: 16 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@ Curious about the general timeline for pyOpenSci reviews?
6666
:link-type: doc
6767
:class-card: left-aligned
6868

69-
Read about our peer review policies.
69+
Read through our peer review policies before submitting a package to us.
7070
:::
7171
::::
7272

@@ -87,9 +87,10 @@ Before you begin this process, [please be sure to read the review process guidel
8787
8888
1. Please be sure that you have time to devote to making changes to your
8989
package. During review, you will receive feedback from an editor and two reviewers. Changes could
90-
take time. Please consider this before submitting to us. You can read more about the timeline to make changes in our [peer review policies page](../our-process/policies).
90+
take time. Please consider this before submitting to us. You can read more about the timeline for making changes on our [peer review policies page](../our-process/policies).
9191
2. A diverse group of volunteer editors and reviewers leads peer review.
9292
Please be considerate when engaging with everyone online.
93+
3. All participants in our peer review process need to follow the [pyOpenSci Code of Conduct](https://www.pyopensci.org/handbook/CODE_OF_CONDUCT.html). Please review it before submitting to us.
9394
```
9495

9596
## 1. Do you plan to continue to maintain your package?
@@ -129,7 +130,7 @@ emphasis on long-term software maintenance and focuses more on
129130
publication quality and citation/credit.
130131
```
131132

132-
## 2. Does Your Package Meet Packaging Requirements?
133+
## 2. Does your package meet packaging requirements?
133134

134135
Before submitting your project for review with pyOpenSci, make sure that
135136
your package meets all of the requirements listed in the editor checks (see below).
@@ -148,6 +149,11 @@ for Python packaging, including discussions of:
148149
149150
```
150151

152+
:::{important}
153+
Given the increased use of Generative AI tools (LLMs), we have developed a disclosure policy for all packages submitted to pyOpenSci (generative-ai-policy). Please review this before submitting to us.
154+
:::
155+
156+
151157
```{hint}
152158
**Do you have questions about Python packaging or our peer review process?**
153159
@@ -170,7 +176,7 @@ that you can use to create a high-quality
170176
Python package.](https://www.pyopensci.org/python-package-guide/package-structure-code/python-package-build-tools.html)
171177
```
172178

173-
## 3. Is Your Package in Scope for pyOpenSci?
179+
## 3. Is your package in scope for pyOpenSci?
174180

175181
Next, check to see if your package falls within the topical and technical scope of pyOpenSci. If you aren't
176182
sure about whether your package fits within pyOpenSci's scope (below), submit
@@ -188,13 +194,13 @@ Our current categories for determining package scope are below:
188194
Click here to view our technical and domain scope requirements.
189195
```
190196

191-
## 4. Submit Your Package for Peer Review
197+
## 4. Submit your package for peer review
192198

193199
To submit your package for peer review, you can
194200
open an issue in our [pyopensci/software-review repo](https://github.com/pyOpenSci/software-review/issues/new/choose/)
195201
repository and fill out the [Submit Software for Review](https://github.com/pyOpenSci/software-submission/issues/new?template=submit-software-for-review.md) issue template.
196202

197-
## 5. Editor-in-Chief Reviews Package for Scope and Minimal Infrastructure Criteria
203+
## 5. Editor-in-Chief reviews package for scope and minimal infrastructure criteria
198204

199205
Once the issue is opened, our editor-in-chief and an editor from our editorial board will review your submission within
200206
**2 weeks** and respond with next steps. The editor may request that you make updates
@@ -207,7 +213,7 @@ package if it does not fall within our scope.
207213
Click here to view the editor checks that will be used to evaluate your package.
208214
```
209215

210-
## 6. The Review Begins
216+
## 6. The review begins
211217

212218
If your package meets the minimal criteria for being
213219
reviewed, it may then be given to an editor with appropriate domain experience
@@ -217,7 +223,7 @@ issue within **3 weeks**. Reviewers can also open issues in your package reposit
217223
We prefer issues that link back to the review as they document changes made to your
218224
package that were triggered by our review process.
219225

220-
## 7. Response to Reviews
226+
## 7. Response to reviews
221227

222228
You should respond to reviewers’ comments within **2 weeks** of the
223229
last-submitted review. You can make updates to your package at any time. We
@@ -231,7 +237,7 @@ Once the reviewers are happy with the changes that you've made to the package, t
231237
editor will review everything and accept your package into the pyOpenSci ecosystem.
232238
Congratulations! You are almost done!
233239

234-
## My Package is Approved, Now What?
240+
## My package is approved, now what?
235241

236242
Congratulations on being accepted into the pyOpenSci community of maintainers!
237243
Once your package is approved, a few things will happen:
@@ -246,7 +252,7 @@ Once your package is approved, a few things will happen:
246252

247253
If you'd like to submit your package to JOSS, you can do so now. Remember that JOSS will accept our review as theirs, so **you DO NOT need to go through another review**. Read more below.
248254

249-
### Journal of Open Source Software (JOSS) Submission
255+
### Journal of Open Source Software (JOSS) submission
250256

251257
pyOpenSci has a [partnership with JOSS](JOSS), where our review is accepted by JOSS by
252258
default if the package fits into the JOSS scope.

our-process/policies.md

Lines changed: 47 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -3,15 +3,15 @@
33
## Review process guidelines
44

55
pyOpenSci packages are reviewed for quality, fit, scope, documentation, and
6-
usability. The review process is similar to a manuscript review, however, it
6+
usability. The review process is similar to a manuscript review; however, it
77
has a stronger focus on Python packaging best practices.
88

99
Unlike a manuscript review, our peer review process is an ongoing conversation.
10-
Once all major issues and questions are addressed, the review editor will make
11-
a decision to accept, hold, or reject the package.
10+
Once all major issues and questions are addressed, the review editor will decide
11+
to accept, hold, or reject the package.
1212

1313
Rejections are usually done early in the process, before the review process
14-
begins. In rare cases, a package may also not be on-boarded into the pyOpenSci
14+
begins. In rare cases, a package may also not be onboarded into the pyOpenSci
1515
ecosystem after review & revision.
1616

1717
It is ultimately the editor’s decision on whether or not to reject the package
@@ -20,26 +20,27 @@ based on how the reviews are addressed.
2020
## Review communication approach
2121

2222
Communication between authors, reviewers, and editors takes
23-
place on GitHub. You can, however choose to contact the editor by email if
23+
place on GitHub. You can, however, choose to contact the editor by email if
2424
needed.
2525

2626
When submitting a package, please make sure that your GitHub notification
27-
settings are setup to notify you when you receive feedback on the review issue.
27+
settings are turned on for the software-submission repository to notify you
28+
when you receive feedback on a review issue.
2829

2930
## Submitting your package for review in other venues
3031

3132
We recommend submitting your package for review with pyOpenSci before
3233
submitting a software paper describing the package to a journal.
3334

34-
Review feedback may result in major improvements and updates to your package,
35-
including changes that could be break package functionality.
35+
Review feedback may result in significant improvements and updates to your package,
36+
including changes that could break package functionality.
3637

3738
Applying reviewer or editor recommendations to your package can improve your
38-
users' experience with future versions of your package even if your package is
39+
users' experience with future versions of your package, even if your package is
3940
already published on `PyPI` or `conda-forge`.
4041

4142
> Please do not submit your package for review while it or an associated
42-
> manuscript is also under review at another venue, as this may result on
43+
> manuscript is also under review at another venue, as this may result in
4344
> conflicting requests for changes from two sets of reviewers.
4445
4546
### Publication with Journal of Open Source Software (JOSS)
@@ -95,6 +96,33 @@ a conflict of interest if:
9596
In the case where none of the associate editors can serve as editor, an
9697
external guest editor will be recruited to lead the package review.
9798

99+
(generative-ai-policy)=
100+
## Policy for use of generative AI / LLMs
101+
102+
:::{admonition} How this policy was developed
103+
:class: important
104+
105+
The policy below was co-developed by the pyOpenSci community. Its goals are:
106+
107+
* Acknowledgment of and transparency around the widespread use of Generative AI tools (with a focus on Large Language Models (LLMs) in Open Source.
108+
* Protect peer review efficiency: Ensure human review of any LLM-generated contributions to a package to protect editor and reviewer volunteer time in our peer review process.
109+
* Raise awareness of challenges that Generative AI tools present to the scientific (and broader) open source community.
110+
[Please see this GitHub issue for a discussion of the topic.](https://github.com/pyOpenSci/software-peer-review/issues/331)
111+
:::
112+
113+
### Disclosure of generative AI use in pyOpenSci reviewed packages
114+
115+
* When you submit a package to pyOpenSci, please disclose any use of LLMs (Large Language Models) in your package’s generation by checking the appropriate boxes on our software submission form. Disclosure should generally include what parts of your package were developed using LLM tools.
116+
* Please also disclose this use of Generative AI tools in your package's `README.md` file and in any modules where generative AI contributions have been implemented.
117+
* We require that all aspects of your package have been reviewed carefully by a human on your maintainer team. Please ensure all text and code have been carefully checked for bias, bugs, and issues before submitting to pyOpenSci.
118+
* Your acknowledgment of using Generative AI will not impact the success of your submission unless you have blindly copied text and code into your package without careful review and evaluation of its accuracy, and for any systemic bias.
119+
* If the review team (comprised of the editor and reviewers) determines that the code and text in the package are too challenging to review, they can decide to pause and/or discontinue the review following this policy’s guidelines.
120+
121+
Below is the checklist that you will need to respond to in our submission form:
122+
123+
```{include} ../appendices/gen-ai-checklist.md
124+
```
125+
98126
## Review timelines and on-hold reviews
99127

100128
At any time, an author can choose to have their submission put on hold
@@ -133,9 +161,9 @@ including issues, pull requests, and dates of the last release and last commit
133161
to the package repository. Activity is defined as a repository commit, pull
134162
request, or release.
135163

136-
We will flag packages that haven't been updated within a 1 year/ 12 month time
164+
We will flag packages that haven't been updated within a 1-year/12-month time
137165
period based on activity. Packages with no activity after 12 months will be
138-
flagged. At that time, pyOpenSci editorial team member will contact the package
166+
flagged. At that time, a pyOpenSci editorial team member will contact the package
139167
maintainers to evaluate the maintenance status of their package.
140168

141169
(archive-process)=
@@ -150,15 +178,15 @@ ecosystem.
150178
In cases where the community heavily uses a package, we may
151179
collaborate with the community to identify reasonable next steps, such as
152180
assisting in finding a new maintainer. If a solution for the ongoing package
153-
maintenance is not found, the package will be archived within the pyOpenSci
154-
ecosystem.
181+
maintenance of your package is not found, the package will be archived within
182+
the pyOpenSci ecosystem. [See section on archiving below.](package-archive)
155183

156184
If a sub-community decides to fork and maintain the package, we are open to
157185
working with the new maintainers to register the newly forked package within
158186
our ecosystem. The original package will be archived with a link to the new
159187
fork.
160188

161-
We will also add a note to any blogs written that highlight your tool that
189+
We will also add a note to any blogs written that highlight your tool, that
162190
the package is no longer maintained.
163191

164192
### Quality commitment
@@ -176,19 +204,20 @@ in touch with us if they do need to step down from maintaining a tool.
176204
### Requesting Package Removal from the pyOpenSci Ecosystem
177205

178206
In the unlikely scenario that a contributor of a package requests removal of
179-
their package from our ecosystem, we retain the right offer the last / most
207+
their package from our ecosystem, we retain the right to offer the last/most
180208
recently released version of that package in our ecosystem for archival
181209
purposes only.
182210

211+
(package-archive)=
183212
### Archiving a package
184213

185214
If a package appears to be no longer maintained, we will mark it as
186-
archived which moves the package from our
215+
archived, which moves the package from our
187216
[main package listing](https://www.pyopensci.org/python-packages.html#all-packages)
188217
to our [archived packaging](https://www.pyopensci.org/python-packages.html#archived-packages)
189218
listing section.
190219

191-
To archive a pyOpenSci approved package, add the
220+
To archive a pyOpenSci-approved package, add the
192221
[archive label](https://github.com/pyOpenSci/software-submission/issues?q=label%3Aarchived)
193222
to the original review issue. Once this label is applied to the issue, the
194223
website will automatically update to reflect this status. If at any point

0 commit comments

Comments
 (0)