Tokenizer from FileReader example

In this example we shall show you how to get a tokenizer from a FileReader. The FileReader is a convenience class for reading character files. The constructors of this class assume that the default character encoding and the default byte-buffer size are appropriate. To get a tokenizer from a FileReader one should perform the following steps:

  • Create a new FileReader, given the name of the file to read from.
  • Create a new StreamTokenizer that parses the given file reader.
  • Get the next token of the tokenizer, and check if it a String, the end of a line, a number, a word or something else,

as described in the code snippet below.

package com.javacodegeeks.snippets.core;


public class StreamToken {

    public static void main(String[] args) throws Exception {

  FileReader fr = null;

  fr = new FileReader("C:/Users/nikos7/Desktop/output.txt");

  StreamTokenizer st = new StreamTokenizer(fr);


  while (st.nextToken() != StreamTokenizer.TT_EOF) {
switch (st.ttype) {
    case ''':
    case '"':
  System.out.println("String = " + st.sval);
    case StreamTokenizer.TT_EOL:
    case StreamTokenizer.TT_NUMBER:
  System.out.println("Number = " + st.nval);
    case StreamTokenizer.TT_WORD:
  System.out.println("Word = " + st.sval);
  System.out.println("Other = " + (char) st.ttype);



Other = &
Word = copy
Other = ;
Number = 2012.0
Other = ,
Word =

This was an example of how to get a tokenizer from a FileReader in Java.

Byron Kiourtzoglou

Byron is a master software engineer working in the IT and Telecom domains. He is an applications developer in a wide variety of applications/services. He is currently acting as the team leader and technical architect for a proprietary service creation and integration platform for both the IT and Telecom industries in addition to a in-house big data real-time analytics solution. He is always fascinated by SOA, middleware services and mobile development. Byron is co-founder and Executive Editor at Java Code Geeks.
Notify of

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Inline Feedbacks
View all comments
Back to top button