Directory Connection

With the directory connection, you can extract multiple files from one directory. The result is the union of all selected files. If the columns in the files don’t match, the output structure contains all columns of all files; the column value is empty for columns that are not contained in a particular file.

Settings

Location

List of locations (more info in article Locations for file-based Connections):

  • FileSystem
  • FTP
  • OneDrive
  • WebDAV
  • WebHDFS
Directory name Full path to the directory containing the files.
File pattern Regular expression on the filenames for the selection of files in the directory.
CSV settings Header: If set, the entries of the first line are used as column headers.
Data delimiter: Separators between the columns in the text file, e.g. ",".
  • Tab as data delimiter: \t
  • For blank value (" ") as data delimiter: #space
  • semicolon
  • comma

Any other string is also valid.

Enclosure character: The enclosure character of the columns. Possible enclosure characters are:
  • #none (no enclosures)
  • double quotes (")
  • single quotes (')

If no enclosure character is defined, by default double quotes (") is used.

Encoding The most prevalent character encodings are UTF-8 (default), ASCII, and latin1 (Windows standard).

A list of all character codes can be found at http://docs.oracle.com/javase/8/docs/technotes/guides/intl/encoding.doc.html
From this list you can also enter manually a character code into the field ‘Encoding’.

Use escape character If set, the character “\” invokes an alternative interpretation of the subsequent character.

Example: “1\”Hello\”2” is interpreted as 1”Hello”2
Add filename column None: no information on the name of the file is available in a file extract.
Name: a file extract on this connection gets an additional column "#_Filename" with the name of the file.
Path: a file extract on this connection gets an additional column "#_Filename" with the name of the file including the full path.
Recursive If set, all sub-directories of the directory are also considered.

Updated June 5, 2023