Skip to content

Commit 6d24cf7

Browse files
jsonBackupjsonBackup
authored
Merge branch 'comfyanonymous:master' into dev
2 parents d1439aa + 11a2ad5 commit 6d24cf7

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

57 files changed

+1929
-584
lines changed

‎.github/workflows/windows_release_nightly_pytorch.yml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ on:
77
description: 'cuda version'
88
required: true
99
type: string
10-
default: "121"
10+
default: "124"
1111

1212
python_minor:
1313
description: 'python minor version'
@@ -19,7 +19,7 @@ on:
1919
description: 'python patch version'
2020
required: true
2121
type: string
22-
default: "2"
22+
default: "3"
2323
# push:
2424
# branches:
2525
# - master
@@ -49,7 +49,7 @@ jobs:
4949
echo 'import site' >> ./python3${{ inputs.python_minor }}._pth
5050
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
5151
./python.exe get-pip.py
52-
python -m pip wheel torch torchvision torchaudio mpmath==1.3.0 --pre --extra-index-url https://download.pytorch.org/whl/nightly/cu${{ inputs.cu }} -r ../ComfyUI/requirements.txt pygit2 -w ../temp_wheel_dir
52+
python -m pip wheel torch torchvision torchaudio mpmath==1.3.0 numpy==1.26.4 --pre --extra-index-url https://download.pytorch.org/whl/nightly/cu${{ inputs.cu }} -r ../ComfyUI/requirements.txt pygit2 -w ../temp_wheel_dir
5353
ls ../temp_wheel_dir
5454
./python.exe -s -m pip install --pre ../temp_wheel_dir/*
5555
sed -i '1i../ComfyUI' ./python3${{ inputs.python_minor }}._pth

‎README.md‎

Lines changed: 46 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -41,29 +41,32 @@ Workflow examples can be found on the [Examples page](https://comfyanonymous.git
4141

4242
## Shortcuts
4343

44-
| Keybind | Explanation |
45-
|---------------------------|--------------------------------------------------------------------------------------------------------------------|
46-
| Ctrl + Enter | Queue up current graph for generation |
47-
| Ctrl + Shift + Enter | Queue up current graph as first for generation |
48-
| Ctrl + Z/Ctrl + Y | Undo/Redo |
49-
| Ctrl + S | Save workflow |
50-
| Ctrl + O | Load workflow |
51-
| Ctrl + A | Select all nodes |
52-
| Alt + C | Collapse/uncollapse selected nodes |
53-
| Ctrl + M | Mute/unmute selected nodes |
54-
| Ctrl + B | Bypass selected nodes (acts like the node was removed from the graph and the wires reconnected through) |
55-
| Delete/Backspace | Delete selected nodes |
56-
| Ctrl + Delete/Backspace | Delete the current graph |
57-
| Space | Move the canvas around when held and moving the cursor |
58-
| Ctrl/Shift + Click | Add clicked node to selection |
59-
| Ctrl + C/Ctrl + V | Copy and paste selected nodes (without maintaining connections to outputs of unselected nodes) |
60-
| Ctrl + C/Ctrl + Shift + V | Copy and paste selected nodes (maintaining connections from outputs of unselected nodes to inputs of pasted nodes) |
61-
| Shift + Drag | Move multiple selected nodes at the same time |
62-
| Ctrl + D | Load default graph |
63-
| Q | Toggle visibility of the queue |
64-
| H | Toggle visibility of history |
65-
| R | Refresh graph |
66-
| Double-Click LMB | Open node quick search palette |
44+
| Keybind | Explanation |
45+
|------------------------------------|--------------------------------------------------------------------------------------------------------------------|
46+
| Ctrl + Enter | Queue up current graph for generation |
47+
| Ctrl + Shift + Enter | Queue up current graph as first for generation |
48+
| Ctrl + Z/Ctrl + Y | Undo/Redo |
49+
| Ctrl + S | Save workflow |
50+
| Ctrl + O | Load workflow |
51+
| Ctrl + A | Select all nodes |
52+
| Alt + C | Collapse/uncollapse selected nodes |
53+
| Ctrl + M | Mute/unmute selected nodes |
54+
| Ctrl + B | Bypass selected nodes (acts like the node was removed from the graph and the wires reconnected through) |
55+
| Delete/Backspace | Delete selected nodes |
56+
| Ctrl + Delete/Backspace | Delete the current graph |
57+
| Space | Move the canvas around when held and moving the cursor |
58+
| Ctrl/Shift + Click | Add clicked node to selection |
59+
| Ctrl + C/Ctrl + V | Copy and paste selected nodes (without maintaining connections to outputs of unselected nodes) |
60+
| Ctrl + C/Ctrl + Shift + V | Copy and paste selected nodes (maintaining connections from outputs of unselected nodes to inputs of pasted nodes) |
61+
| Shift + Drag | Move multiple selected nodes at the same time |
62+
| Ctrl + D | Load default graph |
63+
| Alt + `+` | Canvas Zoom in |
64+
| Alt + `-` | Canvas Zoom out |
65+
| Ctrl + Shift + LMB + Vertical drag | Canvas Zoom in/out |
66+
| Q | Toggle visibility of the queue |
67+
| H | Toggle visibility of history |
68+
| R | Refresh graph |
69+
| Double-Click LMB | Open node quick search palette |
6770

6871
Ctrl can also be replaced with Cmd instead for macOS users
6972

@@ -99,7 +102,7 @@ Put your VAE in: models/vae
99102
### AMD GPUs (Linux only)
100103
AMD users can install rocm and pytorch with pip if you don't have it already installed, this is the command to install the stable version:
101104

102-
```pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm5.7```
105+
```pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm6.0```
103106

104107
This is the command to install the nightly with ROCm 6.0 which might have some performance improvements:
105108

@@ -133,7 +136,16 @@ After this you should have everything installed and can proceed to running Comfy
133136

134137
### Others:
135138

136-
#### [Intel Arc](https://github.com/comfyanonymous/ComfyUI/discussions/476)
139+
#### Intel GPUs
140+
141+
Intel GPU support is available for all Intel GPUs supported by Intel's Extension for Pytorch (IPEX) with the support requirements listed in the [Installation](https://intel.github.io/intel-extension-for-pytorch/index.html#installation?platform=gpu) page. Choose your platform and method of install and follow the instructions. The steps are as follows:
142+
143+
1. Start by installing the drivers or kernel listed or newer in the Installation page of IPEX linked above for Windows and Linux if needed.
144+
1. Follow the instructions to install [Intel's oneAPI Basekit](https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit-download.html) for your platform.
145+
1. Install the packages for IPEX using the instructions provided in the Installation page for your platform.
146+
1. Follow the [ComfyUI manual installation](#manual-install-windows-linux) instructions for Windows and Linux and run ComfyUI normally as described above after everything is installed.
147+
148+
Additional discussion and help can be found [here](https://github.com/comfyanonymous/ComfyUI/discussions/476).
137149

138150
#### Apple Mac silicon
139151

@@ -142,7 +154,7 @@ You can install ComfyUI in Apple Mac silicon (M1 or M2) with any recent macOS ve
142154
1. Install pytorch nightly. For instructions, read the [Accelerated PyTorch training on Mac](https://developer.apple.com/metal/pytorch/) Apple Developer guide (make sure to install the latest pytorch nightly).
143155
1. Follow the [ComfyUI manual installation](#manual-install-windows-linux) instructions for Windows and Linux.
144156
1. Install the ComfyUI [dependencies](#dependencies). If you have another Stable Diffusion UI [you might be able to reuse the dependencies](#i-already-have-another-ui-for-stable-diffusion-installed-do-i-really-have-to-install-all-of-these-dependencies).
145-
1. Launch ComfyUI by running `python main.py --force-fp16`. Note that --force-fp16 will only work if you installed the latest pytorch nightly.
157+
1. Launch ComfyUI by running `python main.py`
146158

147159
> **Note**: Remember to add your models, VAE, LoRAs etc. to the corresponding Comfy folders, as discussed in [ComfyUI manual installation](#manual-install-windows-linux).
148160
@@ -195,20 +207,20 @@ To use a textual inversion concepts/embeddings in a text prompt put them in the
195207
```embedding:embedding_filename.pt```
196208

197209

198-
## How to increase generation speed?
199-
200-
Make sure you use the regular loaders/Load Checkpoint node to load checkpoints. It will auto pick the right settings depending on your GPU.
201-
202-
You can set this command line setting to disable the upcasting to fp32 in some cross attention operations which will increase your speed. Note that this will very likely give you black images on SD2.x models. If you use xformers or pytorch attention this option does not do anything.
203-
204-
```--dont-upcast-attention```
205-
206210
## How to show high-quality previews?
207211

208212
Use ```--preview-method auto``` to enable previews.
209213

210214
The default installation includes a fast latent preview method that's low-resolution. To enable higher-quality previews with [TAESD](https://github.com/madebyollin/taesd), download the [taesd_decoder.pth](https://github.com/madebyollin/taesd/raw/main/taesd_decoder.pth) (for SD1.x and SD2.x) and [taesdxl_decoder.pth](https://github.com/madebyollin/taesd/raw/main/taesdxl_decoder.pth) (for SDXL) models and place them in the `models/vae_approx` folder. Once they're installed, restart ComfyUI to enable high-quality previews.
211215

216+
## How to use TLS/SSL?
217+
Generate a self-signed certificate (not appropriate for shared/production use) and key by running the command: `openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -sha256 -days 3650 -nodes -subj "/C=XX/ST=StateName/L=CityName/O=CompanyName/OU=CompanySectionName/CN=CommonNameOrHostname"`
218+
219+
Use `--tls-keyfile key.pem --tls-certfile cert.pem` to enable TLS/SSL, the app will now be accessible with `https://...` instead of `http://...`.
220+
221+
> Note: Windows users can use [alexisrolland/docker-openssl](https://github.com/alexisrolland/docker-openssl) or one of the [3rd party binary distributions](https://wiki.openssl.org/index.php/Binaries) to run the command example above.
222+
<br/><br/>If you use a container, note that the volume mount `-v` can be a relative path so `... -v ".\:/openssl-certs" ...` would create the key & cert files in the current directory of your command prompt or powershell terminal.
223+
212224
## Support and dev channel
213225

214226
[Matrix space: #comfyui_space:matrix.org](https://app.element.io/#/room/%23comfyui_space%3Amatrix.org) (it's like discord but open source).

‎comfy/cldm/cldm.py‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,7 @@ def __init__(
5252
adm_in_channels=None,
5353
transformer_depth_middle=None,
5454
transformer_depth_output=None,
55+
attn_precision=None,
5556
device=None,
5657
operations=comfy.ops.disable_weight_init,
5758
**kwargs,
@@ -202,7 +203,7 @@ def __init__(
202203
SpatialTransformer(
203204
ch, num_heads, dim_head, depth=num_transformers, context_dim=context_dim,
204205
disable_self_attn=disabled_sa, use_linear=use_linear_in_transformer,
205-
use_checkpoint=use_checkpoint, dtype=self.dtype, device=device, operations=operations
206+
use_checkpoint=use_checkpoint, attn_precision=attn_precision, dtype=self.dtype, device=device, operations=operations
206207
)
207208
)
208209
self.input_blocks.append(TimestepEmbedSequential(*layers))
@@ -262,7 +263,7 @@ def __init__(
262263
mid_block += [SpatialTransformer( # always uses a self-attn
263264
ch, num_heads, dim_head, depth=transformer_depth_middle, context_dim=context_dim,
264265
disable_self_attn=disable_middle_self_attn, use_linear=use_linear_in_transformer,
265-
use_checkpoint=use_checkpoint, dtype=self.dtype, device=device, operations=operations
266+
use_checkpoint=use_checkpoint, attn_precision=attn_precision, dtype=self.dtype, device=device, operations=operations
266267
),
267268
ResBlock(
268269
ch,

‎comfy/cli_args.py‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,8 @@ def __call__(self, parser, namespace, values, option_string=None):
3535

3636
parser.add_argument("--listen", type=str, default="127.0.0.1", metavar="IP", nargs="?", const="0.0.0.0", help="Specify the IP address to listen on (default: 127.0.0.1). If --listen is provided without an argument, it defaults to 0.0.0.0. (listens on all)")
3737
parser.add_argument("--port", type=int, default=8188, help="Set the listen port.")
38+
parser.add_argument("--tls-keyfile", type=str, help="Path to TLS (SSL) key file. Enables TLS, makes app accessible at https://... requires --tls-certfile to function")
39+
parser.add_argument("--tls-certfile", type=str, help="Path to TLS (SSL) certificate file. Enables TLS, makes app accessible at https://... requires --tls-keyfile to function")
3840
parser.add_argument("--enable-cors-header", type=str, default=None, metavar="ORIGIN", nargs="?", const="*", help="Enable CORS (Cross-Origin Resource Sharing) with optional origin or allow all with default '*'.")
3941
parser.add_argument("--max-upload-size", type=float, default=100, help="Set the maximum upload size in MB.")
4042

@@ -49,7 +51,6 @@ def __call__(self, parser, namespace, values, option_string=None):
4951
cm_group.add_argument("--cuda-malloc", action="store_true", help="Enable cudaMallocAsync (enabled by default for torch 2.0 and up).")
5052
cm_group.add_argument("--disable-cuda-malloc", action="store_true", help="Disable cudaMallocAsync.")
5153

52-
parser.add_argument("--dont-upcast-attention", action="store_true", help="Disable upcasting of attention. Can boost speed but increase the chances of black images.")
5354

5455
fp_group = parser.add_mutually_exclusive_group()
5556
fp_group.add_argument("--force-fp32", action="store_true", help="Force fp32 (If this makes your GPU work better please report it).")
@@ -94,6 +95,11 @@ class LatentPreviewMethod(enum.Enum):
9495

9596
parser.add_argument("--disable-xformers", action="store_true", help="Disable xformers.")
9697

98+
upcast = parser.add_mutually_exclusive_group()
99+
upcast.add_argument("--force-upcast-attention", action="store_true", help="Force enable attention upcasting, please report if it fixes black images.")
100+
upcast.add_argument("--dont-upcast-attention", action="store_true", help="Disable all upcasting of attention. Should be unnecessary except for debugging.")
101+
102+
97103
vram_group = parser.add_mutually_exclusive_group()
98104
vram_group.add_argument("--gpu-only", action="store_true", help="Store and run everything (text encoders/CLIP models, etc... on the GPU).")
99105
vram_group.add_argument("--highvram", action="store_true", help="By default models will be unloaded to CPU memory after being used. This option keeps them in GPU memory.")

‎comfy/controlnet.py‎

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -138,11 +138,13 @@ def control_merge(self, control_input, control_output, control_prev, output_dtyp
138138
return out
139139

140140
class ControlNet(ControlBase):
141-
def __init__(self, control_model, global_average_pooling=False, device=None, load_device=None, manual_cast_dtype=None):
141+
def __init__(self, control_model=None, global_average_pooling=False, device=None, load_device=None, manual_cast_dtype=None):
142142
super().__init__(device)
143143
self.control_model = control_model
144144
self.load_device = load_device
145-
self.control_model_wrapped = comfy.model_patcher.ModelPatcher(self.control_model, load_device=load_device, offload_device=comfy.model_management.unet_offload_device())
145+
if control_model is not None:
146+
self.control_model_wrapped = comfy.model_patcher.ModelPatcher(self.control_model, load_device=load_device, offload_device=comfy.model_management.unet_offload_device())
147+
146148
self.global_average_pooling = global_average_pooling
147149
self.model_sampling_current = None
148150
self.manual_cast_dtype = manual_cast_dtype
@@ -183,7 +185,9 @@ def get_control(self, x_noisy, t, cond, batched_number):
183185
return self.control_merge(None, control, control_prev, output_dtype)
184186

185187
def copy(self):
186-
c = ControlNet(self.control_model, global_average_pooling=self.global_average_pooling, load_device=self.load_device, manual_cast_dtype=self.manual_cast_dtype)
188+
c = ControlNet(None, global_average_pooling=self.global_average_pooling, load_device=self.load_device, manual_cast_dtype=self.manual_cast_dtype)
189+
c.control_model = self.control_model
190+
c.control_model_wrapped = self.control_model_wrapped
187191
self.copy_to(c)
188192
return c
189193

0 commit comments

Comments
 (0)