Skip to content

Conversation

@LogicFan
Copy link
Collaborator

@LogicFan LogicFan commented Nov 12, 2025

Contributor: Yongda Fan (yongdaf2@illinois.edu)

Contribution Type: Interpretability, Tests, Example Script

Description
Implemented Gradient Interaction Modifications (GIM) interpreter for StageNet aligned with PyHealth 2.0 interpretability APIs, including temperature-scaled softmax gradients per https://arxiv.org/pdf/2505.17630.
Wired StageNet-compatible hook context so GIM reuses DeepLIFT hooks or dedicated GIM hooks to manage activation overrides safely.
Added focused unit tests covering gradient equivalence, hook routing, temporal tuple inputs, and temperature-based behavior.
Created a MIMIC-IV example that loads StageNet checkpoints, runs GIM attribution, and prints top-ranked features.

Files to Review
pyhealth/interpret/methods/gim.py
pyhealth/interpret/methods/init.py
tests/core/test_gim.py
examples/gim_stagenet_mimic4.py

@LogicFan LogicFan marked this pull request as ready for review November 12, 2025 08:20
Copy link
Collaborator

@jhnwu3 jhnwu3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some quick thoughts that I think would be really cool for an extended showcase as I do want to see if creating an attention-based StageNet would improve interpretability here.

  1. Can we extend this approach to be able to interpret the transformer class here? https://github.com/sunlabuiuc/PyHealth/blob/master/pyhealth/models/transformer.py. Mainly because GIM was intended to also help interpret attention-based models.
  2. Can you update the docs in 2 locations here to incorporate this method? https://github.com/sunlabuiuc/PyHealth/blob/master/docs/api/interpret.rst and add a GIM.rst file in https://github.com/sunlabuiuc/PyHealth/tree/master/docs/api/interpret
  3. On the topic of docs, can we add an example initialization and call in the doc strings? This way users can easily know how to directly use this method.

@LogicFan
Copy link
Collaborator Author

Docs has been updated.

The example for transformer has been created, though i cannot test it as i don't have a transformer checkpoint available.

@LogicFan LogicFan requested a review from jhnwu3 November 18, 2025 03:09
Copy link
Collaborator

@jhnwu3 jhnwu3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, I can test the transformer model on my end later, and implement a fix if need be.

@jhnwu3 jhnwu3 merged commit cf59e1b into sunlabuiuc:master Nov 18, 2025
1 check passed
dalloliogm pushed a commit to dalloliogm/PyHealth that referenced this pull request Nov 26, 2025
* Initial attempts for GIM

* Add testcase and notebook

* Fix incorrect implemenation, and add comments

* Add docs

* Examples in Docs

* Adapt GIM for transformer

---------

Co-authored-by: John Wu <54558896+jhnwu3@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants