fix(fingerprint): treat TMPDIR as strict API and fail (Issue #7877)

[ada-ggf25]

Fixes #7877 and follows up on feedback from #7890.

Context

In #7890 it was introduced an automatic creation of the TMPDIR directory in _TempCacheDir so that datasets does not silently fall back to /tmp when TMPDIR points to a non‑existent path. This addressed the original “tempfile silently ignores TMPDIR” issue in #7877.

During review, it was pointed out by @stas00 (see review comment) that if the code treats TMPDIR as part of the public API, then failures to use it should fail loudly, not just emit warnings, because warnings are easy to miss in complex multi‑GPU setups.

This PR implements that stricter behaviour.

What this PR changes

_TempCacheDir behaviour In src/datasets/fingerprint.py:

• Continue to:

  • Detect TMPDIR from the environment
  • Normalise the path
  • Auto‑create the directory when it does not exist
  • Pass the (validated) directory explicitly to tempfile.mkdtemp(...) so TMPDIR is honoured even if tempfile.gettempdir() was already cached

• New behaviour (in response to review on #7890):

  • If TMPDIR is set, but the directory cannot be created, we now re‑raise an OSError with a clear, actionable message:
    • “TMPDIR is set to '…' but the directory does not exist and could not be created: … Please create it manually or unset TMPDIR to fall back to the default temporary directory.”
  • If TMPDIR is set but points to something that is not a directory, we also raise OSError with guidance:
    • “TMPDIR is set to '…' but it is not a directory. Please point TMPDIR to a writable directory or unset it to fall back to the default temporary directory.”
  • When TMPDIR is not set, behaviour is unchanged: we pass dir=None and let tempfile use the system default temp directory.

This aligns with the @stas00’s suggestion that TMPDIR should be treated as a strict API contract: if the user chooses a TMPDIR, we either use it or clearly fail, rather than silently falling back.

Tests

In tests/test_fingerprint.py:

• Updated tests that previously expected warning‑and‑fallback to now expect hard failures:

  • test_temp_cache_dir_tmpdir_creation_failure
    • Uses unittest.mock.patch to force os.makedirs to raise OSError("Permission denied")
    • Asserts that constructing _TempCacheDir() raises OSError and that the message contains both “TMPDIR is set to” and “could not be created”
  • test_temp_cache_dir_tmpdir_not_directory
    • Points TMPDIR to a regular file and asserts that _TempCacheDir() raises OSError with a message mentioning “is not a directory”

• Left the positive‑path tests in place:

  • test_temp_cache_dir_with_tmpdir_nonexistent – verifies that a non‑existent TMPDIR is created and used
  • test_temp_cache_dir_with_tmpdir_existing – verifies that an existing TMPDIR directory is used as the base for the temp cache dir
  • test_temp_cache_dir_without_tmpdir – verifies behaviour when TMPDIR is not set (default temp directory)

• Kept the earlier fix to test_fingerprint_in_multiprocessing, which now uses Pool.map and asserts that fingerprints are stable across processes.

Rationale

• Treating TMPDIR as part of the API for cache placement means:
  • Users can rely on it to move large temporary Arrow files away from small /tmp partitions.
  • Misconfigured TMPDIR should be immediately visible as a hard error, not as a warning lost among many logs.
• The stricter failure mode matches the concern on #7890 that “warnings are very easy to miss in complex applications where there are already dozens of warnings multiplied by multiple GPU processes”.

Testing

• pytest tests/test_fingerprint.py
• make style
• No new linter issues introduced.