interfaces.io¶
BIDSDataGrabber¶
BIDS datagrabber module that wraps around pybids to allow arbitrary querying of BIDS datasets.
Examples¶
By default, the BIDSDataGrabber fetches anatomical and functional images from a project, and makes BIDS entities (e.g. subject) available for filtering outputs.
>>> bg = BIDSDataGrabber()
>>> bg.inputs.base_dir = 'ds005/'
>>> bg.inputs.subject = '01'
>>> results = bg.run()
Dynamically created, user-defined output fields can also be defined to return different types of outputs from the same project. All outputs are filtered on common entities, which can be explicitly defined as infields.
>>> bg = BIDSDataGrabber(infields = ['subject'], outfields = ['dwi'])
>>> bg.inputs.base_dir = 'ds005/'
>>> bg.inputs.subject = '01'
>>> bg.inputs.output_query['dwi'] = dict(modality='dwi')
>>> results = bg.run()
Inputs:
[Mandatory]
base_dir: (an existing directory name)
Path to BIDS Directory.
[Optional]
raise_on_empty: (a boolean, nipype default value: True)
Generate exception if list is empty for a given field
return_type: ('file' or 'namedtuple', nipype default value: file)
strict: (a boolean)
Return only BIDS "proper" files (e.g., ignore derivatives/,
sourcedata/, etc.)
output_query: (a dictionary with keys which are a unicode string and
with values which are a dictionary with keys which are any value
and with values which are any value)
Queries for outfield outputs
Outputs:
None
DataFinder¶
Search for paths that match a given regular expression. Allows a less proscriptive approach to gathering input files compared to DataGrabber. Will recursively search any subdirectories by default. This can be limited with the min/max depth options. Matched paths are available in the output ‘out_paths’. Any named groups of captured text from the regular expression are also available as ouputs of the same name.
Examples¶
>>> from nipype.interfaces.io import DataFinder
>>> df = DataFinder()
>>> df.inputs.root_paths = '.'
>>> df.inputs.match_regex = '.+/(?P<series_dir>.+(qT1|ep2d_fid_T1).+)/(?P<basename>.+)\.nii.gz'
>>> result = df.run()
>>> result.outputs.out_paths
['./027-ep2d_fid_T1_Gd4/acquisition.nii.gz',
'./018-ep2d_fid_T1_Gd2/acquisition.nii.gz',
'./016-ep2d_fid_T1_Gd1/acquisition.nii.gz',
'./013-ep2d_fid_T1_pre/acquisition.nii.gz']
>>> result.outputs.series_dir
['027-ep2d_fid_T1_Gd4',
'018-ep2d_fid_T1_Gd2',
'016-ep2d_fid_T1_Gd1',
'013-ep2d_fid_T1_pre']
>>> result.outputs.basename
['acquisition',
'acquisition'
'acquisition',
'acquisition']
Inputs:
[Mandatory]
root_paths: (a list of items which are any value or a unicode string)
[Optional]
max_depth: (an integer (int or long))
The maximum depth to search beneath the root_paths
match_regex: (a unicode string, nipype default value: (.+))
Regular expression for matching paths.
ignore_regexes: (a list of items which are any value)
List of regular expressions, if any match the path it will be
ignored.
min_depth: (an integer (int or long))
The minimum depth to search beneath the root paths
unpack_single: (a boolean, nipype default value: False)
Unpack single results from list
Outputs:
None
DataGrabber¶
Generic datagrabber module that wraps around glob in an intelligent way for neuroimaging tasks to grab files
Attention
Doesn’t support directories currently
Examples¶
>>> from nipype.interfaces.io import DataGrabber
Pick all files from current directory
>>> dg = DataGrabber()
>>> dg.inputs.template = '*'
Pick file foo/foo.nii from current directory
>>> dg.inputs.template = '%s/%s.dcm'
>>> dg.inputs.template_args['outfiles']=[['dicomdir','123456-1-1.dcm']]
Same thing but with dynamically created fields
>>> dg = DataGrabber(infields=['arg1','arg2'])
>>> dg.inputs.template = '%s/%s.nii'
>>> dg.inputs.arg1 = 'foo'
>>> dg.inputs.arg2 = 'foo'
however this latter form can be used with iterables and iterfield in a pipeline.
Dynamically created, user-defined input and output fields
>>> dg = DataGrabber(infields=['sid'], outfields=['func','struct','ref'])
>>> dg.inputs.base_directory = '.'
>>> dg.inputs.template = '%s/%s.nii'
>>> dg.inputs.template_args['func'] = [['sid',['f3','f5']]]
>>> dg.inputs.template_args['struct'] = [['sid',['struct']]]
>>> dg.inputs.template_args['ref'] = [['sid','ref']]
>>> dg.inputs.sid = 's1'
Change the template only for output field struct. The rest use the general template
>>> dg.inputs.field_template = dict(struct='%s/struct.nii')
>>> dg.inputs.template_args['struct'] = [['sid']]
Inputs:
[Mandatory]
template: (a unicode string)
Layout used to get files. relative to base directory if defined
sort_filelist: (a boolean)
Sort the filelist that matches the template
[Optional]
drop_blank_outputs: (a boolean, nipype default value: False)
Remove ``None`` entries from output lists
template_args: (a dictionary with keys which are a unicode string and
with values which are a list of items which are a list of items
which are any value)
Information to plug into template
raise_on_empty: (a boolean, nipype default value: True)
Generate exception if list is empty for a given field
base_directory: (an existing directory name)
Path to the base directory consisting of subject data.
Outputs:
None
DataSink¶
Generic datasink module to store structured outputs
Primarily for use within a workflow. This interface allows arbitrary creation of input attributes. The names of these attributes define the directory structure to create for storage of the files or directories.
The attributes take the following form:
string[[.[@]]string[[.[@]]string]] …
where parts between [] are optional.
An attribute such as contrasts.@con will create a ‘contrasts’ directory to store the results linked to the attribute. If the @ is left out, such as in ‘contrasts.con’, a subdirectory ‘con’ will be created under ‘contrasts’.
the general form of the output is:
'base_directory/container/parameterization/destloc/filename'
destloc = string[[.[@]]string[[.[@]]string]] and
filename comesfrom the input to the connect statement.
Warning
This is not a thread-safe node because it can write to a common shared location. It will not complain when it overwrites a file.
Note
If both substitutions and regexp_substitutions are used, then substitutions are applied first followed by regexp_substitutions.
This interface cannot be used in a MapNode as the inputs are defined only when the connect statement is executed.
Examples¶
>>> ds = DataSink()
>>> ds.inputs.base_directory = 'results_dir'
>>> ds.inputs.container = 'subject'
>>> ds.inputs.structural = 'structural.nii'
>>> setattr(ds.inputs, 'contrasts.@con', ['cont1.nii', 'cont2.nii'])
>>> setattr(ds.inputs, 'contrasts.alt', ['cont1a.nii', 'cont2a.nii'])
>>> ds.run()
To use DataSink in a MapNode, its inputs have to be defined at the time the interface is created.
>>> ds = DataSink(infields=['contasts.@con'])
>>> ds.inputs.base_directory = 'results_dir'
>>> ds.inputs.container = 'subject'
>>> ds.inputs.structural = 'structural.nii'
>>> setattr(ds.inputs, 'contrasts.@con', ['cont1.nii', 'cont2.nii'])
>>> setattr(ds.inputs, 'contrasts.alt', ['cont1a.nii', 'cont2a.nii'])
>>> ds.run()
Inputs:
[Optional]
strip_dir: (a directory name)
path to strip out of filename
_outputs: (a dictionary with keys which are a unicode string and with
values which are any value, nipype default value: {})
remove_dest_dir: (a boolean, nipype default value: False)
remove dest directory when copying dirs
creds_path: (a unicode string)
Filepath to AWS credentials file for S3 bucket access; if not
specified, the credentials will be taken from the AWS_ACCESS_KEY_ID
and AWS_SECRET_ACCESS_KEY environment variables
bucket: (any value)
Boto3 S3 bucket for manual override of bucket
encrypt_bucket_keys: (a boolean)
Flag indicating whether to use S3 server-side AES-256 encryption
substitutions: (a list of items which are a tuple of the form: (a
unicode string, a unicode string))
List of 2-tuples reflecting string to substitute and string to
replace it with
container: (a unicode string)
Folder within base directory in which to store output
regexp_substitutions: (a list of items which are a tuple of the form:
(a unicode string, a unicode string))
List of 2-tuples reflecting a pair of a Python regexp pattern and a
replacement string. Invoked after string `substitutions`
local_copy: (a unicode string)
Copy files locally as well as to S3 bucket
base_directory: (a directory name)
Path to the base directory for storing data.
parameterization: (a boolean, nipype default value: True)
store output in parametrized structure
Outputs:
out_file: (any value)
datasink output
FreeSurferSource¶
Generates freesurfer subject info from their directories
Examples¶
>>> from nipype.interfaces.io import FreeSurferSource
>>> fs = FreeSurferSource()
>>> #fs.inputs.subjects_dir = '.'
>>> fs.inputs.subject_id = 'PWS04'
>>> res = fs.run()
>>> fs.inputs.hemi = 'lh'
>>> res = fs.run()
Inputs:
[Mandatory]
subject_id: (a unicode string)
Subject name for whom to retrieve data
subjects_dir: (an existing directory name)
Freesurfer subjects directory.
[Optional]
hemi: ('both' or 'lh' or 'rh', nipype default value: both)
Selects hemisphere specific outputs
Outputs:
entorhinal_exvivo_stats: (a list of items which are an existing file
name)
Entorhinal exvivo statistics files
graymid: (a list of items which are an existing file name)
Graymid/midthickness surface meshes
pial: (a list of items which are an existing file name)
Gray matter/pia mater surface meshes
label: (a list of items which are an existing file name)
Volume and surface label files
ribbon: (a list of items which are an existing file name)
Volumetric maps of cortical ribbons
wm: (an existing file name)
Segmented white-matter volume
orig: (an existing file name)
Base image conformed to Freesurfer space
aparc_aseg: (a list of items which are an existing file name)
Aparc parcellation projected into aseg volume
sulc: (a list of items which are an existing file name)
Surface maps of sulcal depth
T1: (an existing file name)
Intensity normalized whole-head volume
sphere: (a list of items which are an existing file name)
Spherical surface meshes
avg_curv: (a list of items which are an existing file name)
Average atlas curvature, sampled to subject
aparc_a2009s_stats: (a list of items which are an existing file name)
Aparc a2009s parcellation statistics files
area_pial: (a list of items which are an existing file name)
Mean area of triangles each vertex on the pial surface is associated
with
curv_pial: (a list of items which are an existing file name)
Curvature of pial surface
inflated: (a list of items which are an existing file name)
Inflated surface meshes
norm: (an existing file name)
Normalized skull-stripped volume
brain: (an existing file name)
Intensity normalized brain-only volume
white: (a list of items which are an existing file name)
White/gray matter surface meshes
nu: (an existing file name)
Non-uniformity corrected whole-head volume
wmparc_stats: (a list of items which are an existing file name)
White matter parcellation statistics file
aseg: (an existing file name)
Volumetric map of regions from automatic segmentation
jacobian_white: (a list of items which are an existing file name)
Distortion required to register to spherical atlas
volume: (a list of items which are an existing file name)
Surface maps of cortical volume
brainmask: (an existing file name)
Skull-stripped (brain-only) volume
sphere_reg: (a list of items which are an existing file name)
Spherical registration file
rawavg: (an existing file name)
Volume formed by averaging input images
thickness: (a list of items which are an existing file name)
Surface maps of cortical thickness
curv_stats: (a list of items which are an existing file name)
Curvature statistics files
aseg_stats: (a list of items which are an existing file name)
Automated segmentation statistics file
wmparc: (an existing file name)
Aparc parcellation projected into subcortical white matter
smoothwm: (a list of items which are an existing file name)
Smoothed original surface meshes
BA_stats: (a list of items which are an existing file name)
Brodmann Area statistics files
filled: (an existing file name)
Subcortical mass volume
aparc_stats: (a list of items which are an existing file name)
Aparc parcellation statistics files
annot: (a list of items which are an existing file name)
Surface annotation files
curv: (a list of items which are an existing file name)
Maps of surface curvature
JSONFileGrabber¶
Datagrabber interface that loads a json file and generates an output for every first-level object
Example¶
>>> import pprint
>>> from nipype.interfaces.io import JSONFileGrabber
>>> jsonSource = JSONFileGrabber()
>>> jsonSource.inputs.defaults = {'param1': 'overrideMe', 'param3': 1.0}
>>> res = jsonSource.run()
>>> pprint.pprint(res.outputs.get())
{'param1': 'overrideMe', 'param3': 1.0}
>>> jsonSource.inputs.in_file = os.path.join(datadir, 'jsongrabber.txt')
>>> res = jsonSource.run()
>>> pprint.pprint(res.outputs.get())
{'param1': 'exampleStr', 'param2': 4, 'param3': 1.0}
Inputs:
[Optional]
in_file: (an existing file name)
JSON source file
defaults: (a dictionary with keys which are any value and with values
which are any value)
JSON dictionary that sets default outputvalues, overridden by values
found in in_file
Outputs:
None
JSONFileSink¶
Very simple frontend for storing values into a JSON file. Entries already existing in in_dict will be overridden by matching entries dynamically added as inputs.
Warning
This is not a thread-safe node because it can write to a common shared location. It will not complain when it overwrites a file.
>>> jsonsink = JSONFileSink(input_names=['subject_id', ... 'some_measurement']) >>> jsonsink.inputs.subject_id = 's1' >>> jsonsink.inputs.some_measurement = 11.4 >>> jsonsink.run()Using a dictionary as input:
>>> dictsink = JSONFileSink() >>> dictsink.inputs.in_dict = {'subject_id': 's1', ... 'some_measurement': 11.4} >>> dictsink.run()
Inputs:
[Optional]
in_dict: (a dictionary with keys which are any value and with values
which are any value, nipype default value: {})
input JSON dictionary
_outputs: (a dictionary with keys which are any value and with values
which are any value, nipype default value: {})
out_file: (a file name)
JSON sink file
Outputs:
out_file: (a file name)
JSON sink file
MySQLSink¶
Very simple frontend for storing values into MySQL database.
Examples¶
>>> sql = MySQLSink(input_names=['subject_id', 'some_measurement'])
>>> sql.inputs.database_name = 'my_database'
>>> sql.inputs.table_name = 'experiment_results'
>>> sql.inputs.username = 'root'
>>> sql.inputs.password = 'secret'
>>> sql.inputs.subject_id = 's1'
>>> sql.inputs.some_measurement = 11.4
>>> sql.run()
Inputs:
[Mandatory]
table_name: (a unicode string)
database_name: (a unicode string)
Otherwise known as the schema name
config: (a file name)
MySQL Options File (same format as my.cnf)
mutually_exclusive: host
host: (a unicode string, nipype default value: localhost)
mutually_exclusive: config
requires: username, password
[Optional]
username: (a unicode string)
password: (a unicode string)
Outputs:
None
S3DataGrabber¶
Generic datagrabber module that wraps around glob in an intelligent way for neuroimaging tasks to grab files from Amazon S3
Works exactly like DataGrabber, except, you must specify an S3 “bucket” and “bucket_path” to search for your data and a “local_directory” to store the data. “local_directory” should be a location on HDFS for Spark jobs. Additionally, “template” uses regex style formatting, rather than the glob-style found in the original DataGrabber.
Inputs:
[Mandatory]
template: (a unicode string)
Layout used to get files. Relative to bucket_path if defined.Uses
regex rather than glob style formatting.
sort_filelist: (a boolean)
Sort the filelist that matches the template
bucket: (a unicode string)
Amazon S3 bucket where your data is stored
[Optional]
local_directory: (an existing directory name)
Path to the local directory for subject data to be downloaded and
accessed. Should be on HDFS for Spark jobs.
template_args: (a dictionary with keys which are a unicode string and
with values which are a list of items which are a list of items
which are any value)
Information to plug into template
bucket_path: (a unicode string, nipype default value: )
Location within your bucket for subject data.
raise_on_empty: (a boolean, nipype default value: True)
Generate exception if list is empty for a given field
region: (a unicode string, nipype default value: us-east-1)
Region of s3 bucket
anon: (a boolean, nipype default value: False)
Use anonymous connection to s3. If this is set to True, boto may
print a urlopen error, but this does not prevent data from being
downloaded.
Outputs:
None
SQLiteSink¶
Very simple frontend for storing values into SQLite database.
Warning
This is not a thread-safe node because it can write to a common shared location. It will not complain when it overwrites a file.
Examples¶
>>> sql = SQLiteSink(input_names=['subject_id', 'some_measurement'])
>>> sql.inputs.database_file = 'my_database.db'
>>> sql.inputs.table_name = 'experiment_results'
>>> sql.inputs.subject_id = 's1'
>>> sql.inputs.some_measurement = 11.4
>>> sql.run()
Inputs:
[Mandatory]
database_file: (an existing file name)
table_name: (a unicode string)
Outputs:
None
SSHDataGrabber¶
Extension of DataGrabber module that downloads the file list and optionally the files from a SSH server. The SSH operation must not need user and password so an SSH agent must be active in where this module is being run.
Attention
Doesn’t support directories currently
Examples¶
>>> from nipype.interfaces.io import SSHDataGrabber
>>> dg = SSHDataGrabber()
>>> dg.inputs.hostname = 'test.rebex.net'
>>> dg.inputs.user = 'demo'
>>> dg.inputs.password = 'password'
>>> dg.inputs.base_directory = 'pub/example'
Pick all files from the base directory
>>> dg.inputs.template = '*'
Pick all files starting with “s” and a number from current directory
>>> dg.inputs.template_expression = 'regexp'
>>> dg.inputs.template = 'pop[0-9].*'
Same thing but with dynamically created fields
>>> dg = SSHDataGrabber(infields=['arg1','arg2'])
>>> dg.inputs.hostname = 'test.rebex.net'
>>> dg.inputs.user = 'demo'
>>> dg.inputs.password = 'password'
>>> dg.inputs.base_directory = 'pub'
>>> dg.inputs.template = '%s/%s.txt'
>>> dg.inputs.arg1 = 'example'
>>> dg.inputs.arg2 = 'foo'
however this latter form can be used with iterables and iterfield in a pipeline.
Dynamically created, user-defined input and output fields
>>> dg = SSHDataGrabber(infields=['sid'], outfields=['func','struct','ref'])
>>> dg.inputs.hostname = 'myhost.com'
>>> dg.inputs.base_directory = '/main_folder/my_remote_dir'
>>> dg.inputs.template_args['func'] = [['sid',['f3','f5']]]
>>> dg.inputs.template_args['struct'] = [['sid',['struct']]]
>>> dg.inputs.template_args['ref'] = [['sid','ref']]
>>> dg.inputs.sid = 's1'
Change the template only for output field struct. The rest use the general template
>>> dg.inputs.field_template = dict(struct='%s/struct.nii')
>>> dg.inputs.template_args['struct'] = [['sid']]
Inputs:
[Mandatory]
template: (a unicode string)
Layout used to get files. relative to base directory if defined
hostname: (a unicode string)
Server hostname.
base_directory: (a unicode string)
Path to the base directory consisting of subject data.
sort_filelist: (a boolean)
Sort the filelist that matches the template
[Optional]
drop_blank_outputs: (a boolean, nipype default value: False)
Remove ``None`` entries from output lists
template_expression: ('fnmatch' or 'regexp', nipype default value:
fnmatch)
Use either fnmatch or regexp to express templates
download_files: (a boolean, nipype default value: True)
If false it will return the file names without downloading them
ssh_log_to_file: (a unicode string, nipype default value: )
If set SSH commands will be logged to the given file
username: (a unicode string)
Server username.
template_args: (a dictionary with keys which are a unicode string and
with values which are a list of items which are a list of items
which are any value)
Information to plug into template
raise_on_empty: (a boolean, nipype default value: True)
Generate exception if list is empty for a given field
password: (a string)
Server password.
Outputs:
None
SelectFiles¶
Flexibly collect data from disk to feed into workflows.
This interface uses the {}-based string formatting syntax to plug values (possibly known only at workflow execution time) into string templates and collect files from persistant storage. These templates can also be combined with glob wildcards. The field names in the formatting template (i.e. the terms in braces) will become inputs fields on the interface, and the keys in the templates dictionary will form the output fields.
Examples¶
>>> import pprint
>>> from nipype import SelectFiles, Node
>>> templates={"T1": "{subject_id}/struct/T1.nii",
... "epi": "{subject_id}/func/f[0, 1].nii"}
>>> dg = Node(SelectFiles(templates), "selectfiles")
>>> dg.inputs.subject_id = "subj1"
>>> pprint.pprint(dg.outputs.get()) # doctest:
{'T1': <undefined>, 'epi': <undefined>}
The same thing with dynamic grabbing of specific files:
>>> templates["epi"] = "{subject_id}/func/f{run!s}.nii"
>>> dg = Node(SelectFiles(templates), "selectfiles")
>>> dg.inputs.subject_id = "subj1"
>>> dg.inputs.run = [2, 4]
Inputs:
[Optional]
raise_on_empty: (a boolean, nipype default value: True)
Raise an exception if a template pattern matches no files.
force_lists: (a boolean or a list of items which are a unicode
string, nipype default value: False)
Whether to return outputs as a list even when only one file matches
the template. Either a boolean that applies to all output fields or
a list of output field names to coerce to a list
base_directory: (an existing directory name)
Root path common to templates.
sort_filelist: (a boolean, nipype default value: True)
When matching mutliple files, return them in sorted order.
Outputs:
None
XNATSink¶
Generic datasink module that takes a directory containing a list of nifti files and provides a set of structured output fields.
Inputs:
[Mandatory]
subject_id: (a unicode string)
Set to subject id
project_id: (a unicode string)
Project in which to store the outputs
config: (a file name)
mutually_exclusive: server
server: (a unicode string)
mutually_exclusive: config
requires: user, pwd
experiment_id: (a unicode string)
Set to workflow name
[Optional]
_outputs: (a dictionary with keys which are a unicode string and with
values which are any value, nipype default value: {})
reconstruction_id: (a unicode string)
Option to customize ouputs representation in XNAT - reconstruction
level will be used with specified id
mutually_exclusive: assessor_id
user: (a unicode string)
share: (a boolean, nipype default value: False)
Option to share the subjects from the original projectinstead of
creating new ones when possible - the created experiments are then
shared back to the original project
pwd: (a string)
assessor_id: (a unicode string)
Option to customize ouputs representation in XNAT - assessor level
will be used with specified id
mutually_exclusive: reconstruction_id
cache_dir: (a directory name)
Outputs:
None
XNATSource¶
Generic XNATSource module that wraps around the pyxnat module in an intelligent way for neuroimaging tasks to grab files and data from an XNAT server.
Examples¶
>>> from nipype.interfaces.io import XNATSource
Pick all files from current directory
>>> dg = XNATSource()
>>> dg.inputs.template = '*'
>>> dg = XNATSource(infields=['project','subject','experiment','assessor','inout'])
>>> dg.inputs.query_template = '/projects/%s/subjects/%s/experiments/%s' '/assessors/%s/%s_resources/files'
>>> dg.inputs.project = 'IMAGEN'
>>> dg.inputs.subject = 'IMAGEN_000000001274'
>>> dg.inputs.experiment = '*SessionA*'
>>> dg.inputs.assessor = '*ADNI_MPRAGE_nii'
>>> dg.inputs.inout = 'out'
>>> dg = XNATSource(infields=['sid'],outfields=['struct','func'])
>>> dg.inputs.query_template = '/projects/IMAGEN/subjects/%s/experiments/*SessionA*' '/assessors/*%s_nii/out_resources/files'
>>> dg.inputs.query_template_args['struct'] = [['sid','ADNI_MPRAGE']]
>>> dg.inputs.query_template_args['func'] = [['sid','EPI_faces']]
>>> dg.inputs.sid = 'IMAGEN_000000001274'
Inputs:
[Mandatory]
query_template: (a unicode string)
Layout used to get files. Relative to base directory if defined
config: (a file name)
mutually_exclusive: server
server: (a unicode string)
mutually_exclusive: config
requires: user, pwd
[Optional]
pwd: (a string)
query_template_args: (a dictionary with keys which are a unicode
string and with values which are a list of items which are a list
of items which are any value, nipype default value: {'outfiles':
[]})
Information to plug into template
cache_dir: (a directory name)
Cache directory
user: (a unicode string)
Outputs:
None
copytree()¶
Recursively copy a directory tree using nipype.utils.filemanip.copyfile()
This is not a thread-safe routine. However, in the case of creating new directories, it checks to see if a particular directory has already been created by another process.