Merge remote-tracking branch 'nipype_proper/master' into FreeSurferBreakout

David Ellis · David Ellis · commit 4b850c83914b · 2016-04-01T08:33:20.000-05:00
diff --git a/.travis.yml b/.travis.yml
@@ -34,6 +34,7 @@ before_install:
   echo 'include_dirs = /usr/include:/usr/include/X11' >> $HOME/.numpy-site.cfg;
   fi
 install:
+- sudo apt-get install fusefat
 - conda update --yes conda
 - conda create -n testenv --yes pip python=$TRAVIS_PYTHON_VERSION
 - source activate testenv
@@ -56,7 +57,7 @@ install:
 - pip install -r requirements.txt  # finish remaining requirements
 - python setup.py install
 script:
-- python -W once:FSL:UserWarning:nipype `which nosetests` --with-doctest --with-cov --cover-package nipype --cov-config .coveragerc --logging-level=INFO
+- python -W once:FSL:UserWarning:nipype `which nosetests` --with-doctest --with-cov --cover-package nipype --cov-config .coveragerc --logging-level=DEBUG --verbosity=3
 after_success:
 - coveralls --config_file .coveragerc
 deploy:
diff --git a/CHANGES b/CHANGES
@@ -24,6 +24,7 @@ Next release
 * ENH: New interfaces for interacting with AWS S3: S3DataSink and S3DataGrabber (https://github.com/nipy/nipype/pull/1201)
 * ENH: Interfaces for MINC tools (https://github.com/nipy/nipype/pull/1304)
 * FIX: Use realpath to determine hard link source (https://github.com/nipy/nipype/pull/1388)
+* FIX: Correct linking/copying fallback behavior (https://github.com/nipy/nipype/pull/1391)
 * ENH: Nipype workflow and interfaces for FreeSurfer's recon-all (https://github.com/nipy/nipype/pull/1326)
 
 Release 0.11.0 (September 15, 2015)
diff --git a/CHANGES.orig b/CHANGES.orig
diff --git a/README.rst b/README.rst
@@ -34,6 +34,10 @@ NIPYPE: Neuroimaging in Python: Pipelines and Interfaces
     :target: https://pypi.python.org/pypi/nipype/
     :alt: License
 
+.. image:: https://img.shields.io/badge/gitter-join%20chat%20%E2%86%92-brightgreen.svg?style=flat
+    :target: http://gitter.im/nipy/nipype
+    :alt: Chat
+
 Current neuroimaging software offer users an incredible opportunity to
 analyze data using a variety of different algorithms. However, this has
 resulted in a heterogeneous collection of specialized applications
diff --git a/circle.yml b/circle.yml
@@ -12,7 +12,7 @@ dependencies:
     - bash <(wget -q -O- http://neuro.debian.net/_files/neurodebian-travis.sh)
   override:
     # Install apt packages
-    - sudo apt-get install -y fsl-core fsl-atlases fsl-mni152-templates fsl-feeds afni swig python-vtk xvfb
+    - sudo apt-get install -y fsl-core fsl-atlases fsl-mni152-templates fsl-feeds afni swig python-vtk xvfb fusefat
     - echo 'source /etc/fsl/fsl.sh' >> $HOME/.profile
     - echo 'source /etc/afni/afni.sh' >> $HOME/.profile
     - mkdir -p ~/examples/ && ln -sf /usr/share/fsl-feeds/ ~/examples/feeds
diff --git a/examples/README b/examples/README
@@ -1,4 +1,4 @@
 A dataset for use with these scripts can be downloaded from the nipype
 website. At the time of writing, it's at:
 
-http://nipy.sourceforge.net/nipype/users/pipeline_tutorial.html
+http://nipy.org/nipype/users/pipeline_tutorial.html
diff --git a/nipype/interfaces/petpvc.py b/nipype/interfaces/petpvc.py
@@ -6,81 +6,8 @@
    >>> filepath = os.path.dirname( os.path.realpath( __file__ ) )
    >>> datadir = os.path.realpath(os.path.join(filepath, '../testing/data'))
    >>> os.chdir(datadir)
-
-Nipype interface for PETPVC.
-
-PETPVC is a software from the Nuclear Medicine Department
-of the UCL University Hospital, London, UK.
-
-Its source code is here: https://github.com/UCL/PETPVC
-
-The methods that it implement are explained here:
-K. Erlandsson, I. Buvat, P. H. Pretorius, B. A. Thomas, and B. F. Hutton,
-"A review of partial volume correction techniques for emission tomography
-and their applications in neurology, cardiology and oncology," Phys. Med.
-Biol., vol. 57, no. 21, p. R119, 2012.
-
-There is a publication waiting to be accepted for this software tool.
-
-
-Its command line help shows this:
-
-   -i --input < filename >
-      = PET image file
-   -o --output < filename >
-      = Output file
-   [ -m --mask < filename > ]
-      = Mask image file
-   -p --pvc < keyword >
-      = Desired PVC method
-   -x < X >
-      = The full-width at half maximum in mm along x-axis
-   -y < Y >
-      = The full-width at half maximum in mm along y-axis
-   -z < Z >
-      = The full-width at half maximum in mm along z-axis
-   [ -d --debug ]
-      = Prints debug information
-   [ -n --iter [ Val ] ]
-      = Number of iterations
-        With: Val (Default = 10)
-   [ -k [ Val ] ]
-      = Number of deconvolution iterations
-        With: Val (Default = 10)
-   [ -a --alpha [ aval ] ]
-      = Alpha value
-        With: aval (Default = 1.5)
-   [ -s --stop [ stopval ] ]
-      = Stopping criterion
-        With: stopval (Default = 0.01)
-
-----------------------------------------------
-Technique - keyword
-
-Geometric transfer matrix - "GTM"
-Labbe approach - "LABBE"
-Richardson-Lucy - "RL"
-Van-Cittert - "VC"
-Region-based voxel-wise correction - "RBV"
-RBV with Labbe - "LABBE+RBV"
-RBV with Van-Cittert - "RBV+VC"
-RBV with Richardson-Lucy - "RBV+RL"
-RBV with Labbe and Van-Cittert - "LABBE+RBV+VC"
-RBV with Labbe and Richardson-Lucy- "LABBE+RBV+RL"
-Multi-target correction - "MTC"
-MTC with Labbe - "LABBE+MTC"
-MTC with Van-Cittert - "MTC+VC"
-MTC with Richardson-Lucy - "MTC+RL"
-MTC with Labbe and Van-Cittert - "LABBE+MTC+VC"
-MTC with Labbe and Richardson-Lucy- "LABBE+MTC+RL"
-Iterative Yang - "IY"
-Iterative Yang with Van-Cittert - "IY+VC"
-Iterative Yang with Richardson-Lucy - "IY+RL"
-Muller Gartner - "MG"
-Muller Gartner with Van-Cittert - "MG+VC"
-Muller Gartner with Richardson-Lucy - "MG+RL"
-
 """
+
 from __future__ import print_function
 from __future__ import division
 
@@ -144,6 +71,76 @@ class PETPVCOutputSpec(TraitedSpec):
 class PETPVC(CommandLine):
     """ Use PETPVC for partial volume correction of PET images.
 
+    PETPVC is a software from the Nuclear Medicine Department
+    of the UCL University Hospital, London, UK.
+
+    Its source code is here: https://github.com/UCL/PETPVC
+
+    The methods that it implement are explained here:
+    K. Erlandsson, I. Buvat, P. H. Pretorius, B. A. Thomas, and B. F. Hutton,
+    "A review of partial volume correction techniques for emission tomography
+    and their applications in neurology, cardiology and oncology," Phys. Med.
+    Biol., vol. 57, no. 21, p. R119, 2012.
+
+    There is a publication waiting to be accepted for this software tool.
+
+    Its command line help shows this:
+
+       -i --input < filename >
+          = PET image file
+       -o --output < filename >
+          = Output file
+       [ -m --mask < filename > ]
+          = Mask image file
+       -p --pvc < keyword >
+          = Desired PVC method
+       -x < X >
+          = The full-width at half maximum in mm along x-axis
+       -y < Y >
+          = The full-width at half maximum in mm along y-axis
+       -z < Z >
+          = The full-width at half maximum in mm along z-axis
+       [ -d --debug ]
+          = Prints debug information
+       [ -n --iter [ Val ] ]
+          = Number of iterations
+            With: Val (Default = 10)
+       [ -k [ Val ] ]
+          = Number of deconvolution iterations
+            With: Val (Default = 10)
+       [ -a --alpha [ aval ] ]
+          = Alpha value
+            With: aval (Default = 1.5)
+       [ -s --stop [ stopval ] ]
+          = Stopping criterion
+            With: stopval (Default = 0.01)
+
+    ----------------------------------------------
+    Technique - keyword
+
+    Geometric transfer matrix - "GTM"
+    Labbe approach - "LABBE"
+    Richardson-Lucy - "RL"
+    Van-Cittert - "VC"
+    Region-based voxel-wise correction - "RBV"
+    RBV with Labbe - "LABBE+RBV"
+    RBV with Van-Cittert - "RBV+VC"
+    RBV with Richardson-Lucy - "RBV+RL"
+    RBV with Labbe and Van-Cittert - "LABBE+RBV+VC"
+    RBV with Labbe and Richardson-Lucy- "LABBE+RBV+RL"
+    Multi-target correction - "MTC"
+    MTC with Labbe - "LABBE+MTC"
+    MTC with Van-Cittert - "MTC+VC"
+    MTC with Richardson-Lucy - "MTC+RL"
+    MTC with Labbe and Van-Cittert - "LABBE+MTC+VC"
+    MTC with Labbe and Richardson-Lucy- "LABBE+MTC+RL"
+    Iterative Yang - "IY"
+    Iterative Yang with Van-Cittert - "IY+VC"
+    Iterative Yang with Richardson-Lucy - "IY+RL"
+    Muller Gartner - "MG"
+    Muller Gartner with Van-Cittert - "MG+VC"
+    Muller Gartner with Richardson-Lucy - "MG+RL"
+
     Examples
     --------
     >>> from ..testing import example_data
diff --git a/nipype/interfaces/tests/test_utility.py b/nipype/interfaces/tests/test_utility.py
@@ -63,7 +63,6 @@ def increment_array(in_array):
     f2 = pe.MapNode(utility.Function(input_names=['in_array'], output_names=['out_array'], function=increment_array), name='increment_array', iterfield=['in_array'])
 
     wf.connect(f1, 'random_array', f2, 'in_array')
-
     wf.run()
 
     # Clean up
@@ -167,3 +166,57 @@ def test_csvReader():
                 yield assert_equal, out.outputs.column_1, ['hello', 'world', 'goodbye']
                 yield assert_equal, out.outputs.column_2, ['300.1', '5', '0.3']
         os.unlink(name)
+
+
+def test_aux_connect_function():
+    """ This tests excution nodes with multiple inputs and auxiliary
+    function inside the Workflow connect function.
+    """
+    tempdir = os.path.realpath(mkdtemp())
+    origdir = os.getcwd()
+    os.chdir(tempdir)
+
+    wf = pe.Workflow(name="test_workflow")
+
+    def _gen_tuple(size):
+        return [1, ] * size
+
+    def _sum_and_sub_mul(a, b, c):
+        return (a+b)*c, (a-b)*c
+
+    def _inc(x):
+        return x + 1
+
+    params = pe.Node(utility.IdentityInterface(fields=['size', 'num']), name='params')
+    params.inputs.num  = 42
+    params.inputs.size = 1
+
+    gen_tuple = pe.Node(utility.Function(input_names=['size'],
+                                         output_names=['tuple'],
+                                         function=_gen_tuple),
+                                         name='gen_tuple')
+
+    ssm = pe.Node(utility.Function(input_names=['a', 'b', 'c'],
+                                   output_names=['sum', 'sub'],
+                                   function=_sum_and_sub_mul),
+                                   name='sum_and_sub_mul')
+
+    split = pe.Node(utility.Split(splits=[1, 1],
+                                  squeeze=True),
+                    name='split')
+
+
+    wf.connect([
+                (params,    gen_tuple,  [(("size", _inc),   "size")]),
+                (params,    ssm,        [(("num", _inc),    "c")]),
+                (gen_tuple, split,      [("tuple",          "inlist")]),
+                (split,     ssm,        [(("out1", _inc),   "a"),
+                                         ("out2",           "b"),
+                                        ]),
+                ])
+
+    wf.run()
+
+    # Clean up
+    os.chdir(origdir)
+    shutil.rmtree(tempdir)
diff --git a/nipype/pipeline/engine/workflows.py b/nipype/pipeline/engine/workflows.py
@@ -690,7 +690,7 @@ def _configure_exec_nodes(self, graph):
             node.input_source = {}
             for edge in graph.in_edges_iter(node):
                 data = graph.get_edge_data(*edge)
-                for sourceinfo, field in sorted(data['connect']):
+                for sourceinfo, field in data['connect']:
                     node.input_source[field] = \
                         (op.join(edge[0].output_dir(),
                                  'result_%s.pklz' % edge[0].name),
diff --git a/nipype/testing/__init__.py b/nipype/testing/__init__.py
@@ -29,7 +29,7 @@
 from numpy.testing import *
 
 from . import decorators as dec
-from .utils import skip_if_no_package, package_check
+from .utils import skip_if_no_package, package_check, TempFATFS
 
 skipif = dec.skipif
 
diff --git a/nipype/testing/tests/test_utils.py b/nipype/testing/tests/test_utils.py
@@ -0,0 +1,19 @@
+# emacs: -*- mode: python; py-indent-offset: 4; indent-tabs-mode: nil -*-
+# vi: set ft=python sts=4 ts=4 sw=4 et:
+"""Test testing utilities
+"""
+
+import os
+import warnings
+from nipype.testing.utils import TempFATFS
+from nose.tools import assert_true
+
+
+def test_tempfatfs():
+    try:
+        fatfs = TempFATFS()
+    except IOError:
+        warnings.warn("Cannot mount FAT filesystems with FUSE")
+    else:
+        with fatfs as tmpdir:
+            yield assert_true, os.path.exists(tmpdir)
diff --git a/nipype/testing/utils.py b/nipype/testing/utils.py
@@ -4,6 +4,12 @@
 """
 __docformat__ = 'restructuredtext'
 
+import os
+import time
+import shutil
+import signal
+import subprocess
+from tempfile import mkdtemp
 from ..utils.misc import package_check
 from nose import SkipTest
 
@@ -19,3 +25,66 @@ def skip_if_no_package(*args, **kwargs):
     package_check(exc_failed_import=SkipTest,
                   exc_failed_check=SkipTest,
                   *args, **kwargs)
+
+
+class TempFATFS(object):
+    def __init__(self, size_in_mbytes=8, delay=0.5):
+        """Temporary filesystem for testing non-POSIX filesystems on a POSIX
+        system.
+
+        with TempFATFS() as fatdir:
+            target = os.path.join(fatdir, 'target')
+            copyfile(file1, target, copy=False)
+            assert_false(os.path.islink(target))
+
+        Arguments
+        ---------
+        size_in_mbytes : int
+            Size (in MiB) of filesystem to create
+        delay : float
+            Time (in seconds) to wait for fusefat to start, stop
+        """
+        self.delay = delay
+        self.tmpdir = mkdtemp()
+        self.dev_null = open(os.devnull, 'wb')
+
+        vfatfile = os.path.join(self.tmpdir, 'vfatblock')
+        self.vfatmount = os.path.join(self.tmpdir, 'vfatmount')
+        self.canary = os.path.join(self.vfatmount, '.canary')
+
+        with open(vfatfile, 'wb') as fobj:
+            fobj.write(b'\x00' * (int(size_in_mbytes) << 20))
+        os.mkdir(self.vfatmount)
+
+        mkfs_args = ['mkfs.vfat', vfatfile]
+        mount_args = ['fusefat', '-o', 'rw+', '-f', vfatfile, self.vfatmount]
+
+        subprocess.check_call(args=mkfs_args, stdout=self.dev_null,
+                              stderr=self.dev_null)
+        self.fusefat = subprocess.Popen(args=mount_args, stdout=self.dev_null,
+                                        stderr=self.dev_null)
+        time.sleep(self.delay)
+
+        if self.fusefat.poll() is not None:
+            raise IOError("fatfuse terminated too soon")
+
+        open(self.canary, 'wb').close()
+
+    def __enter__(self):
+        return self.vfatmount
+
+    def __exit__(self, exc_type, exc_val, exc_tb):
+        if self.fusefat is not None:
+            self.fusefat.send_signal(signal.SIGINT)
+
+            # Allow 1s to return without sending terminate
+            for count in range(10):
+                time.sleep(0.1)
+                if self.fusefat.poll() is not None:
+                    break
+            else:
+                self.fusefat.terminate()
+            time.sleep(self.delay)
+            assert not os.path.exists(self.canary)
+        self.dev_null.close()
+        shutil.rmtree(self.tmpdir)
diff --git a/nipype/utils/filemanip.py b/nipype/utils/filemanip.py
diff --git a/nipype/utils/tests/test_filemanip.py b/nipype/utils/tests/test_filemanip.py
