This example will demonstrate iterating over the files and directories recursively from a root directory. The example program will utilize some of the Java NIO features. Central to this example are the DirectoryStream, Filter, Path and Paths classes.
1. Introduction
Before diving into the example program an understanding of the 4 abstractions listed above:
1.1 DirectoryStream
The DirectoryStream interface enables us to iterate over the files and directories within a specific directory. The interface extends Iterable meaning we can use it in a general purpose for loop . While we cannot obtain more than one Iterator when invoking the iterator()method, the hasNext() method is guaranteed to read ahead by at least one element.
This means if it returns truethen an invocation of next()will not throw an IOException. DirectoryStreams are opened when created and closed when invoking the close()method making it a prime candidate for try ... finally idioms or try with resources.
1.2 Filter
A FunctionalInterface type that specifies the criteria under which a directory or file entry should be accepted or filtered out when iterating over the entries for a directory.
1.3 Path
An immutable and thus thread safe abstraction that represents a way to locate a File in a file system. A Path is hierarchial and could contain a root element with many subsequent nested elements and a leaf element representing the name of the file or directory being referred to. (eg: /home/user/Documents). A Path can be registered with a WatchService and watched for changes.
1.4 Paths
The Paths class provides static utility for converting an URI or String to a Path.
2. Technologies used
The example code in this article was built and run using:
- Java 1.8.101 (1.8.x will do fine)
- Maven 3.3.9 (3.3.x will do fine)
- Spring source tool suite 4.6.3 (Any Java IDE would work)
- Ubuntu 16.04 (Windows, Mac or Linux will do fine)
3. Overview
The example program is a console application that can be run with the java -jarcommand once built using the maven instruction mvn clean install package. It makes use of the Apache commons commnad line parser to parse command line arguments when running the application.
To run the application simply navigate to the project root and execute the following in order:
mvn clean install package- navigate to the “target” folder within the project root and execute the following:
java -jar iterate-directory-0.0.1-SNAPSHOT.jar -r <path to root folder> -v -f <filter>where-ris required and-vand-fare optional. The mere presence of-vwill indicate verbose mode and does not require an additional argument whereas-fdoes and will merely filter our any file names that do not contain the literal you supply as argument. For the purposes of this example that was sufficient for demonstrating iteration with filtering, however the filtering can facilitate far more powerful filtering should the need arise.
Examples running the application:
java -jar iterate-directory-0.0.1-SNAPSHOT.jar -r /home/user/temp -vthis will list all the directories and files contained within/home/user/tempin verbose mode. This will mean it will include file creation and modification time stamps and file / directory sizes.java -jar iterate-directory-0.0.1-SNAPSHOT.jar -r /home/user/temp -v -f Testthis will list all the directories contained within/home/user/tempin verbose mode. This will also list the files within each of the nested directories that contain the given pattern in their file names.
4. The code
Listing – Responsible for listing the directory contents
public final class Listing {
private final CLIConfig config;
private final ConsoleView view;
public Listing(final CLIConfig config) {
this.config = config;
this.view = new ConsoleView(config);
}
public void list() {
final Path root = Paths.get(this.config.getRoot().orElseThrow(RuntimeException::new));
if (Files.isDirectory(root)) {
final Directory rootDir = new Directory(root);
list(rootDir, this.config.getFilter());
this.view.render(rootDir);
} else {
throw new RuntimeException("Root needs to be a directory");
}
}
private void list(final Directory directory, final Optional<String> filter) {
assert !Objects.isNull(directory) && !Objects.isNull(filter);
DirectoryStream stream = null;
try {
if (filter.isPresent()) {
stream = Files.newDirectoryStream(directory.getPath(), createFilter(filter.get()));
} else {
stream = Files.newDirectoryStream(directory.getPath());
}
for (Path path : stream) {
if (Files.isDirectory(path)) {
final Directory child = new Directory(path);
directory.addDirectory(child);
list(child, filter);
} else {
directory.addFile(new File(path));
}
}
} catch (IOException e) {
throw new RuntimeException(e);
} finally {
if (!Objects.isNull(stream)) {
try {
stream.close();
} catch (IOException e) {
throw new RuntimeException("unable to close stream while listing directories", e);
}
}
}
}
private Filter<Path> createFilter(final String pattern) {
return new Filter<Path>() {
@Override
public boolean accept(final Path entry) throws IOException {
return Files.isDirectory(entry) || entry.toFile().getName().contains(pattern);
}
};
}
}
- line 12: we call
Paths.get(...)passing the parsed command line argument representing the root directory to begin the listing from. - line 14: it is paramount that our starting point is a directory and we guard against this
- lines 29-33: involve the creation of a DirectoryStream. Depending on if there is a filter pattern provided or not we create a Directory by creating and supplying a Filter or not.
- lines 35-43: we iterate over the Path entries from the DirectoryStream. We check if the Path is a directory
Paths.isDirectory(...)and if it is we create our ownDirectoryproxy to encapsulate the specific Path and recursively iterate again. Should it be a file, we create our ownFileproxy to encapsulate the file and add it to the currentDirectoryproxy we are currently listing. - lines 58-67: we create a Filter that matches all directories and any files that contain the given pattern in their names.
ConsoleView – Responsible for rendering the result of the directory listing
final class ConsoleView {
private final boolean verbose;
ConsoleView(final CLIConfig config) {
this.verbose = config.isVerbose();
}
void render(final Directory directory) {
render(directory, StringUtils.EMPTY);
}
private void render(final Directory directory, final String padding) {
assert !Objects.isNull(directory) && !StringUtils.isNotEmpty(padding);
System.out.println(padding + " d -- " + enrichContent(directory));
directory.getFileChildren().stream().forEach(file -> render(file, padding + "\t"));
directory.getDirectoryChildren().stream().forEach(dir -> render(dir, padding + "\t"));
}
private void render(final File file, final String padding) {
assert !Objects.isNull(file) && !StringUtils.isNotEmpty(padding);
System.out.println(padding + " f -- " + enrichContent(file));
}
private String enrichContent(final Directory directory) {
assert !Objects.isNull(directory);
try {
return this.verbose ? directory.getCreated().toString() + " : " + directory.getModified() + " : " + directory.getSize() + " " + directory.getName()
: directory.getName();
} catch (IOException e) {
return this.verbose ? "E " + directory.getName() : directory.getName();
}
}
private String enrichContent(final File file) {
assert !Objects.isNull(file);
try {
return this.verbose ? file.getCreated() + " : " + file.getModified() + " : " + file.getSize() + " " + file.getName() : file.getName();
} catch (IOException e) {
return this.verbose ? "E " + file.getName() : file.getName();
}
}
}
- Trying to conform to an MVC approach, we only depend on the our domain model,
DirectoryandFilecustom abstractions and help render them via a tree hierarchical view. - Whats important to note is that the Path abstractions are wrapped by our custom domain model and thus ultimately pooled into memory prior to writing them to console. This would mean deep listings might prove challenging on memory requirements, but for this example I suspect the concept is illustrated well enough.
FSNode – FileSystemNode encapsulating shared behaviour between File and Directory domain model
public abstract class FSNode {
private final Path path;
protected FSNode(final Path path) {
Objects.requireNonNull(path);
this.path = path;
}
protected final Path getPath() {
return this.path;
}
protected final String getName() {
return this.path.toFile().getName();
}
protected final long getSize() throws IOException {
return Files.readAttributes(this.path, BasicFileAttributes.class).size();
}
protected final FileTime getCreated() throws IOException {
return Files.readAttributes(this.path, BasicFileAttributes.class).creationTime();
}
protected final FileTime getModified() throws IOException {
return Files.readAttributes(this.path, BasicFileAttributes.class).lastModifiedTime();
}
}
- line 20,24,28: we read the BasicFileAttributes via the
Files.readAttributes(...)method on the Files class. This gives us access to attributes such as date created, date modified, size etc.
5. The output
Sample output from running application in non verbose mode
d -- surefire-reports
f -- TEST-com.javacodegeeks.nio.heapbytebuffer.HeapByteBufferIdentityCrisisTest.xml
f -- TEST-com.javacodegeeks.nio.heapbytebuffer.HeapByteBufferMemoryTest.xml
f -- TEST-com.javacodegeeks.nio.heapbytebuffer.HeapByteBufferReadOnlyTest.xml
f -- com.javacodegeeks.nio.heapbytebuffer.HeapByteBufferReadOnlyTest.txt
f -- com.javacodegeeks.nio.heapbytebuffer.HeapByteBufferMemoryTest.txt
f -- TEST-com.javacodegeeks.nio.heapbytebuffer.HeapByteBufferIdentityTest.xml
f -- com.javacodegeeks.nio.heapbytebuffer.HeapByteBufferIdentityCrisisTest.txt
f -- com.javacodegeeks.nio.heapbytebuffer.HeapByteBufferIdentityTest.txt
d -- maven-archiver
f -- pom.properties
d -- test-classes
d -- com
d -- javacodegeeks
d -- nio
d -- heapbytebuffer
f -- HeapByteBufferIdentityCrisisTest.class
f -- HeapByteBufferReadOnlyTest.class
f -- HeapByteBufferIdentityTest.class
f -- HeapByteBufferMemoryTest.class
dstands for directory andfstands for file- the hierarchical structure is preserved on layout to show the directory hierarchy
Sample output from running application in verbose mode
d -- 2017-06-29T15:18:03Z : 2017-06-29T15:18:03Z : 4096 main
d -- 2017-06-29T15:20:31Z : 2017-06-29T15:20:31Z : 4096 java
d -- 2017-06-29T15:20:31Z : 2017-06-29T15:20:31Z : 4096 com
d -- 2017-06-29T15:20:31Z : 2017-06-29T15:20:31Z : 4096 javacodegeeks
d -- 2017-06-29T15:20:31Z : 2017-06-29T15:20:31Z : 4096 nio
d -- 2017-07-02T07:37:01Z : 2017-07-02T07:37:01Z : 4096 iterate_directory
f -- 2017-07-02T07:40:32Z : 2017-07-02T07:40:32Z : 170 File.java
f -- 2017-07-01T11:57:41Z : 2017-07-01T11:57:41Z : 273 Constants.java
f -- 2017-07-02T07:38:45Z : 2017-07-02T07:38:45Z : 1083 FSNode.java
f -- 2017-07-02T08:36:14Z : 2017-07-02T08:36:14Z : 2548 Listing.java
f -- 2017-07-02T07:39:30Z : 2017-07-02T07:39:30Z : 868 Directory.java
f -- 2017-07-01T11:57:09Z : 2017-07-01T11:57:09Z : 48 package-info.java
f -- 2017-07-02T08:49:04Z : 2017-07-02T08:49:04Z : 1948 ConsoleView.java
d -- 2017-07-01T11:56:10Z : 2017-07-01T11:56:10Z : 4096 runner
- Verbose sample output showing created (first) and modified date (second) with the size of the Path element (directory or file).
6. Summary
In this example we demonstrated how to iterate over the files and directory entries within a specified directory. We covered some Nio abstractions core to this example and working with files and directories in general. These included the DirectoryStream, Filter, Path and Paths abstractions.
7. Download the Source Code
This was a Java Nio Iterate Over Files in Directory example.
You can download the full source code of this example here: Java Nio Iterate Over Files in Directory example
