Merge pull request #20 from unixwzrd/dev

20250621_00 Promote dev to main

Author: unixwzrd

.gitignore

@@ -292,19 +292,24 @@ $RECYCLE.BIN/
 *.msp
 
 # Windows shortcuts
-# !test_automation/logs/*.log
-# !test_automation/logs/transformers/*/*/*
-# !test_automation/reports/*
-# !test_automation/reports/*/*
+.cursor
+.cursorignore
+.cursorrules
+.DS_Store
 .master-planning
 .project-planning
 .project-planning/
 .vscode
+.windsurfrules
 *.bak*
+*.code-workspace
 *.lnk
 *.wav
 *.wav
-*.code-workspace
+# !test_automation/logs/*.log
+# !test_automation/logs/transformers/*/*/*
+# !test_automation/reports/*
+# !test_automation/reports/*/*
 backup/
 bashexp*.txt
 custom-tts/
@@ -318,5 +323,4 @@ tmp
 torchdevice_tts_final_patch.txt
 torchdevice_tts_patch.txt
 torchdevice_tts_pr_patch.txt
-TorchDevice.original
-.DS_Store
+TorchDevice.original/

.windsurfrules

@@ -1,5 +1,17 @@
 # CHANGELOG
 
+## 0.5.2 - 2025-06-21
+
+### Cleanup, Documentation, and Test Results
+
+### Added
+
+- **Test Automation Framework (`test_automation/`)**:
+  - Enhanced `generate_test_report.py` script to improve test file path resolution and clickable links in the Markdown report.
+  - Updated `test_automation/README.md` with instructions for setting up and running the Transformers test suite.
+  - Updated [README](README.md) with a link to the advanced test automation guide.
+  - Updated [CONTRIBUTING](CONTRIBUTING.md) with instructions for running external tests.
+
 ## 0.5.1 - 2025-06-20
 
 ### Improved

20250506-rollback.code-workspace

@@ -111,6 +111,22 @@ When requesting features:
 - Provide examples of how it would be used
 - Discuss potential implementation approaches
 
+## External Project Tests
+
+Feel free to try other projects using TorchDevice. If you find issues, please report them as GitHub issues, or submit a patch with a fix in a PR.  I cannot guarantee that all projects will work, and I may not be able to get to every test project, but I appreciate the assistance in debugging and making this work better for the community.
+
+### PyTorch Transformers
+
+Checkout the [test_automation/README.md](test_automation/README.md) for instructions on running the PyTorch Transformers test suite. For how to manage external projects, see [Managing Test Target Projects](#managing-test-target-projects).
+
+to run the transformers tests, use:
+
+```bash
+time ( for projectgroup in $( (cd test_projects/transformers/tests; /usr/local/bin/ls -l | grep -v .py  | grep -v __ | awk '{print $NF}' ) ); do python test_automation/run_transformer_tests.py --project_root test_projects/transformers $projectgroup; done ) 2>1 | tee output.log
+```
+
+To run the all the tests it takes about 1 hour and 30 minutes.
+
 ## License
 
 By contributing, you agree that your contributions will be licensed under the project's MIT License.

CHANGELOG.md

@@ -6,7 +6,7 @@ A Python library for transparent device handling in PyTorch, enabling seamless t
 
 ## Overview
 
-TorchDevice provides transparent device handling for PyTorch applications, allowing code written for CUDA to run on MPS devices (and vice versa) without modification. It intercepts device-specific calls and redirects them appropriately based on available hardware.
+TorchDevice provides transparent device handling for PyTorch applications, allowing code written for CUDA to run on MPS devices (and vice versa) without modification. It intercepts device-specific calls and redirects them appropriately based on available hardware. This is primarily for migration of existing CUDA code and running existing CUDA code on Apple Silicon Macs.
 
 ## Features
 
@@ -23,20 +23,25 @@ TorchDevice provides transparent device handling for PyTorch applications, allow
 
 ## Installation
 
+Clone and install from the repository:
 ```bash
-pip install TorchDevice
+git clone https://github.com/unixwzrd/TorchDevice.git
+cd TorchDevice
+pip install -e .
 ```
 
 ## Quick Start
 
 ```python
-import TorchDevice  # This automatically hooks into PyTorch
+import TorchDevice  # This automatically hooks into PyTorch, order not important.
 import torch
 
 # Your existing PyTorch code works as is
 model = torch.nn.Linear(10, 10).cuda()  # Will use MPS if CUDA isn't available
 ```
 
+**NOTE:** The project is under active development and produces very verbose logs, so you may want to redirect them to a file or `/dev/null`. I will be working on reducing the verbosity in the future.
+
 ## Project Structure
 
 The project is organized into three main components:
@@ -53,6 +58,7 @@ For detailed structure information, see [Project Structure](docs/project_structu
 - [TorchDevice Behavior](docs/TorchDevice_Behavior.md): Core functionality and behavior
 - [API Reference](docs/TorchDevice_Functions.md): Comprehensive API documentation
 - [CUDA Operations](docs/CUDA-Operations.md): CUDA-specific functionality
+- [Advanced External ProjectTesting](test_automation/README.md): Comprehensive integration testing
 
 ## Development
 
@@ -90,17 +96,39 @@ python run_tests_and_install.py --test-only tests/[test-name.py]
 
 ### Advanced Integration Testing
 
+The most current project files are in the dev branch and may be accessed from there by checking out the project and then checking out the dev branch.
+
+```bash
+# Checkout the project
+git clone https://github.com/yourusername/TorchDevice.git
+
+cd TorchDevice
+
+# Checkout the dev branch
+git checkout dev
+```
+
+If you wish to contribute create a branch and then submit a PR for your changes. I will review them and merge them into the dev branch and then into the main branch as I get to them. I will try to get to them as soon as possible. I appreciate all contributions to making this better.
+
 For comprehensive integration testing against large codebases like Hugging Face Transformers, a dedicated test automation suite is available. This suite manages complex test execution, logging, and reporting. For detailed instructions on setup, dependencies, and usage, please see the dedicated guide:
 
 [**Advanced Test Automation README**](test_automation/README.md)
 
+The latest test reports are in the [test_automation/reports](test_automation/reports) directory. Feel free to have a look and let me know if you have any questions or suggestions.  If you have any fixes, please submit a PR.
+
 ## Contributing
 
 We welcome contributions! Please see our [Contributing Guidelines](CONTRIBUTING.md) for details.
 
+It has additional information on testing other projects with TorchDevice, though I have mostly tested and am working on debugging the issues with HuggingFace Transformers. For now, I am focused on getting the core functionality working.  Right now about 13% of the tests are failing and many of those are due to CUDA specific functionality in the CUDA library.
+
+This is not my primary project I am working on, but I am using it for other projects I am working on, giving me access to available PyTorch code written for CUDA and saving time in having to port code over to Apple Silicon.
+
 ## Supporting This Project
 
-TorchDevice is an open-source project developed and maintained by [Michael P. Sullivan (unixwzrd)](https://unixwzrd.ai). If you find this project useful, please consider supporting its development:
+TorchDevice is an open-source project developed and maintained by [me, M S - unixwzrd](https://unixwzrd.ai). It has evolved and grown over the past 9 months from when it was first conceived, and is now relatively stable and useful for running PyTorch CUDA code on Apple Silicon, and I have tested it with a number of projects. Some projects having to do with Text-to-Speech and other AI/ML projects. My goal right now is to have the HuggingFace Transformers library working with it as much as possible.
+
+If you find this project useful, please consider supporting its development:
 
 ### Funding Options
 - [Patreon](https://www.patreon.com/unixwzrd)

CONTRIBUTING.md

@@ -63,17 +63,22 @@
   - [ ] Implement context managers for these operations
   - [ ] Ensure proper cleanup even in case of exceptions
 
-- [ ] **Implement Lazy Formatting for Log Messages**
-  - [ ] Identify places where f-strings are used for messages that might be filtered out
-  - [ ] Replace these with format strings that are only evaluated if the message will be logged
-  - [ ] Verify that the changes don't affect the log output
-  - [ ] Measure the performance impact of the change
+- [x] **Implement Lazy Formatting for Log Messages**
+  - [x] Identify places where f-strings are used for messages that might be filtered out
+  - [x] Replace these with format strings that are only evaluated if the message will be logged
+  - [x] Verify that the changes don't affect the log output
+  - [x] Measure the performance impact of the change
 
 - [ ] **Remove Unused Features and Code**
   - [ ] Identify unused code or overly complex sections
   - [ ] Simplify or remove these sections
   - [ ] Verify that the changes don't affect the functionality
 
+- [ ] **Reduce Default Logger Verbosity**
+  - [ ] Implement a log level configuration (e.g., via environment variable or function call).
+  - [ ] Change the default log level to be less verbose (e.g., INFO or WARNING).
+  - [ ] Ensure debug-level logging remains available for development.
+
 ## Device Handling Improvements
 
 - [x] **Implement CPU Override Feature**
@@ -90,6 +95,15 @@
   - [ ] Optimize performance for CPU-specific operations
   - [ ] Add support for dynamic device allocation based on operation requirements
 
+## Test Automation & Documentation
+
+- [x] **Create Test Automation Framework**
+  - [x] Enhanced `generate_test_report.py` for better path resolution and clickable links.
+  - [x] Updated `test_automation/README.md` with setup and run instructions.
+- [x] **Update Project Documentation**
+  - [x] Added link to advanced test automation guide in main `README.md`.
+  - [x] Updated `CONTRIBUTING.md` with instructions for running external project tests.
+
 ## Test Framework Improvements
 
 - [ ] **Refine Test Utilities into Proper Modules**