Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
15d779a
init uv project
EmoTim Nov 13, 2025
2186c7b
add weight.npy file
EmoTim Nov 14, 2025
ac7d269
merge main for weight.npy file
EmoTim Nov 14, 2025
1605007
add out dir to gitignore
EmoTim Nov 14, 2025
62bc901
add weight modulation
EmoTim Nov 14, 2025
5bdb02b
add image grid
EmoTim Nov 14, 2025
f32ba50
add vscode debugging config
EmoTim Nov 14, 2025
80d7689
changeb default alpha ranges
EmoTim Nov 14, 2025
9228c05
Merge branch 'timotheschmidt/add-weight-vector' of https://github.com…
EmoTim Nov 14, 2025
965b184
make grid with all images
EmoTim Nov 14, 2025
1fb1339
Merge branch 'timotheschmidt/add-weight-vector' of https://github.com…
EmoTim Nov 14, 2025
e8d87e8
add batching
EmoTim Nov 14, 2025
7cadb6c
Revert "add batching"
EmoTim Nov 14, 2025
6a529ae
add layers selection for adding weight vector
EmoTim Nov 17, 2025
53afa91
add color analysis
EmoTim Nov 17, 2025
16410da
minor uv change
EmoTim Nov 17, 2025
5a45ef1
add packages
EmoTim Nov 17, 2025
78edf85
aggregate results and clearer plots
EmoTim Nov 17, 2025
7230802
Y-axis as relative percentages
EmoTim Nov 17, 2025
4bbb5c8
analysis by style range
EmoTim Nov 17, 2025
e9e0c7f
per channel plots
EmoTim Nov 17, 2025
f989c17
new default alphas
EmoTim Nov 17, 2025
38f13d7
remove large alpha values (10)
EmoTim Nov 17, 2025
04fa3d0
Red Proportion - shows the relative proportion of red in the total RG…
EmoTim Nov 17, 2025
8a98cb9
add proportion plot for every rbg channel
EmoTim Nov 17, 2025
7268b13
Only keep RGB analysis
EmoTim Nov 17, 2025
fb1d303
remove comments
EmoTim Nov 17, 2025
86fffda
show relative brightness change
EmoTim Nov 18, 2025
9213c1f
groupe images by alpha values
EmoTim Nov 18, 2025
7179fd9
Merge branch 'timotheschmidt/add-weight-vector' of https://github.com…
EmoTim Nov 18, 2025
b1a0bd5
alpha patchwork script
EmoTim Nov 18, 2025
21b204f
change ""
EmoTim Nov 18, 2025
548c70a
remove unused variable
EmoTim Nov 18, 2025
11d9e1a
adding the age estimator
EmoTim Nov 18, 2025
2ce9c51
small changes
EmoTim Nov 18, 2025
1ee6a11
Merge branch 'timotheschmidt/add-weight-vector' of https://github.com…
EmoTim Nov 18, 2025
4200143
add onnx dpendency
EmoTim Nov 18, 2025
28326b7
Merge branch 'timotheschmidt/add-weight-vector' of https://github.com…
EmoTim Nov 18, 2025
796b353
uv lock
EmoTim Nov 18, 2025
6cf0ae5
truncation + remove unused file
EmoTim Nov 19, 2025
af6c41c
RGB to BGR
EmoTim Nov 19, 2025
37cb2b5
Merge branch 'timotheschmidt/add-weight-vector' of https://github.com…
EmoTim Nov 19, 2025
f78d1b8
change estimator settings
EmoTim Nov 19, 2025
3cf1048
smaller detection size
EmoTim Nov 19, 2025
af6563b
add warniong suppression
EmoTim Nov 19, 2025
c082d4a
Merge branch 'timotheschmidt/add-weight-vector' of https://github.com…
EmoTim Nov 19, 2025
188b16d
lower verbosity
EmoTim Nov 19, 2025
22ecb78
add vgg pretrained on FFHQ age predictor
EmoTim Nov 19, 2025
0394423
normalize tensor
EmoTim Nov 19, 2025
c74ed62
remove redundant tensor manipulation
EmoTim Nov 19, 2025
a36d5f6
remove obsolete code
EmoTim Nov 19, 2025
c88b62e
first sagemaker setup
EmoTim Nov 19, 2025
c6a60af
Merge branch 'timotheschmidt/add-weight-vector' of https://github.com…
EmoTim Nov 19, 2025
35cc805
put vgg path as arg
EmoTim Nov 19, 2025
a508f94
correct path
EmoTim Nov 19, 2025
3de0a18
update doc
EmoTim Nov 19, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,5 @@
__pycache__/
.cache/
out
.DS_Store
*.pth
1 change: 1 addition & 0 deletions .python-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3.12
14 changes: 14 additions & 0 deletions .vscode/launch.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
"version": "0.2.0",
"configurations": [
{
"name": "Python: Debug Current File",
"type": "debugpy",
"request": "launch",
"program": "${file}",
"console": "integratedTerminal",
"cwd": "${workspaceFolder}",
"justMyCode": true
}
]
}
170 changes: 170 additions & 0 deletions SAGEMAKER_SETUP.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,170 @@
# SageMaker Image Generation Setup Guide

## What You Have Now

✅ **requirements.txt** - All Python dependencies exported from your uv setup
✅ **launch_sagemaker_generation.py** - SageMaker launcher script
✅ **AWS credentials** - Configured via SSO (profile: Timothe)
✅ **S3 bucket** - stylegan2-generated-images_v2
✅ **IAM Role** - arn:aws:iam::183295429625:role/service-role/AmazonSageMaker-ExecutionRole-20241001T141781

## How It Works

1. **ScriptProcessor** uploads your entire repo to SageMaker
2. Installs dependencies from requirements.txt
3. Runs generate.py on GPU instances (ml.g4dn.xlarge)
4. Each instance generates a portion of the 1M images
5. Images are continuously uploaded to S3 as they're generated

## Step-by-Step Instructions

### Step 1: Install SageMaker Python SDK

```bash
uv pip install sagemaker boto3
```

### Step 2: (Optional) Upload Your Trained Model to S3

If you have a custom trained model (not using NVIDIA's pretrained):

```bash
# Upload pre-trained VGG checkpoint to S3
aws s3 cp /home/sagemaker-user/stylegan2-ada-pytorch/dex_age_classifier.pth \
s3://emobot-prod-workspace-bucket/usecases/emobot-research/datasets/stylegan2-generated-images_v2/models/vgg.pth

# Upload weight vector
aws s3 cp /home/sagemaker-user/stylegan2-ada-pytorch/weight.npy \
s3://emobot-prod-workspace-bucket/usecases/emobot-research/datasets/stylegan2-generated-images_v2/models/weight.npy
```

### Step 3: Run a Test Job (RECOMMENDED)

Start with a small test to verify everything works:

```bash
python launch_sagemaker_generation.py --test
```

This will:
- Generate only 100 images
- Use 1 instance
- Run for max 1 hour
- Help verify your setup works

### Step 4: Launch Full Production Job

Once the test succeeds, launch the full job:

```bash
# Using NVIDIA's pretrained FFHQ model (no upload needed)
python launch_sagemaker_generation.py \
--total-images 1000000 \
--instances 10 \
--role arn:aws:iam::183295429625:role/service-role/AmazonSageMaker-ExecutionRole-20241001T141781

# OR with your custom model
python launch_sagemaker_generation.py \
--total-images 1000000 \
--instances 10 \
--role arn:aws:iam::183295429625:role/service-role/AmazonSageMaker-ExecutionRole-20241001T141781 \
--model-s3 s3://stylegan2-generated-images_v2/models/model.pkl

# OR with weight vector modulation
python launch_sagemaker_generation.py \
--total-images 1000000 \
--instances 10 \
--role arn:aws:iam::183295429625:role/service-role/AmazonSageMaker-ExecutionRole-20241001T141781 \
--weight-vector-s3 s3://stylegan2-generated-images_v2/models/weight.npy
```

### Step 5: Monitor Progress

Monitor your job at:
https://console.aws.amazon.com/sagemaker/home?region=eu-west-3#/processing-jobs

Or via CLI:
```bash
aws sagemaker list-processing-jobs --profile Timothe
```

### Step 6: Access Generated Images

Images will be uploaded to:
```
s3://stylegan2-generated-images_v2/generated_images_YYYYMMDD_HHMMSS/
```

Download them:
```bash
aws s3 sync s3://stylegan2-generated-images_v2/generated_images_YYYYMMDD_HHMMSS/ ./downloaded_images/ --profile Timothe
```

## Configuration Options

### Instance Types

- **ml.g4dn.xlarge** (1 GPU, 16 GB RAM) - ~$0.736/hour - Default
- **ml.g4dn.2xlarge** (1 GPU, 32 GB RAM) - ~$0.94/hour - More memory
- **ml.g4dn.4xlarge** (1 GPU, 64 GB RAM) - ~$1.50/hour - If you need more RAM

### Cost Estimation

For 1M images with default settings:
- 10 instances × ml.g4dn.xlarge × 24 hours = ~$176
- Adjust instance_count and max_runtime based on your needs

### Parallelization Strategy

The script splits images evenly across instances:
- **10 instances** → 100K images each
- **20 instances** → 50K images each (faster but more expensive)
- **5 instances** → 200K images each (cheaper but slower)

## Customizing Generation

Edit `launch_sagemaker_generation.py` to customize:

```python
launch_image_generation(
total_images=1_000_000,
instance_count=10,
truncation_psi=0.7, # Adjust truncation
noise_mode='random', # or 'const', 'none'
alphas="-5,0,5", # For weight modulation
style_range=(0, 17), # Style block range
max_runtime_hours=24,
)
```

## Troubleshooting

### Issue: "Could not auto-detect SageMaker role"
**Solution:** Pass the role explicitly with `--role` flag

### Issue: "Access Denied" when uploading to S3
**Solution:** Check your IAM role has S3 write permissions:
```bash
aws iam get-role-policy --role-name AmazonSageMaker-ExecutionRole-20241001T141781 --policy-name S3Access --profile Timothe
```

### Issue: Job fails with "CUDA out of memory"
**Solution:** Reduce batch size or use larger instance (ml.g4dn.2xlarge)

### Issue: Dependencies fail to install
**Solution:** The requirements.txt is comprehensive. If issues persist, you may need to create a custom Docker image.

## Next Steps After Setup

1. ✅ Run test job with `--test` flag
2. ✅ Verify images appear in S3
3. ✅ Launch full production job
4. ✅ Monitor progress in SageMaker console
5. ✅ Download and use generated images

## Important Notes

- **Continuous Upload**: Images upload to S3 as they're generated (no need to wait for job completion)
- **Resumability**: If a job fails, you can restart with different seed ranges
- **Cost Control**: Set `max_runtime_hours` to prevent runaway costs
- **Region**: All resources are in eu-west-3 (Paris)
34 changes: 34 additions & 0 deletions age_estimator.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
import torch
import torch.nn.functional as F
from vgg import VGG


class AgePredictor:
def __init__(self, vgg_path: str):
self.age_net = VGG()
ckpt = torch.load(vgg_path, map_location="cpu")['state_dict']
ckpt = {k.replace('-', '_'): v for k, v in ckpt.items()}
self.age_net.load_state_dict(ckpt)
self.age_net.cuda()
self.age_net.eval()
self.min_age = 0
self.max_age = 100

def __get_predicted_age(self, age_pb):
predict_age_pb = F.softmax(age_pb)
predict_age = torch.zeros(age_pb.size(0)).type_as(predict_age_pb)
for i in range(age_pb.size(0)):
for j in range(age_pb.size(1)):
predict_age[i] += j * predict_age_pb[i][j]
return predict_age

def extract_age(self, x):
x = F.interpolate(x, size=(224, 224), mode='bilinear', align_corners=False) # shape: [1, 3, 224, 224]
predict_age_pb = self.age_net(x)['fc8']
predicted_age = self.__get_predicted_age(predict_age_pb)
return predicted_age

def __call__(self, img: torch.tensor) -> float | None:
"""Allow instance to be called directly."""
return self.extract_age(img)[0]

Binary file added alpha_patch_work.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading