Home » Core Java » io » FileInputStream » Tokenizer from FileReader example

About Byron Kiourtzoglou

Avatar photo
Byron is a master software engineer working in the IT and Telecom domains. He is an applications developer in a wide variety of applications/services. He is currently acting as the team leader and technical architect for a proprietary service creation and integration platform for both the IT and Telecom industries in addition to a in-house big data real-time analytics solution. He is always fascinated by SOA, middleware services and mobile development. Byron is co-founder and Executive Editor at Java Code Geeks.

Tokenizer from FileReader example

In this example we shall show you how to get a tokenizer from a FileReader. The FileReader is a convenience class for reading character files. The constructors of this class assume that the default character encoding and the default byte-buffer size are appropriate. To get a tokenizer from a FileReader one should perform the following steps:

  • Create a new FileReader, given the name of the file to read from.
  • Create a new StreamTokenizer that parses the given file reader.
  • Get the next token of the tokenizer, and check if it a String, the end of a line, a number, a word or something else,

as described in the code snippet below.

package com.javacodegeeks.snippets.core;

import java.io.FileReader;
import java.io.StreamTokenizer;

public class StreamToken {

    public static void main(String[] args) throws Exception {

  FileReader fr = null;

  fr = new FileReader("C:/Users/nikos7/Desktop/output.txt");

  StreamTokenizer st = new StreamTokenizer(fr);


  while (st.nextToken() != StreamTokenizer.TT_EOF) {
switch (st.ttype) {
    case ''':
    case '"':
  System.out.println("String = " + st.sval);
    case StreamTokenizer.TT_EOL:
    case StreamTokenizer.TT_NUMBER:
  System.out.println("Number = " + st.nval);
    case StreamTokenizer.TT_WORD:
  System.out.println("Word = " + st.sval);
  System.out.println("Other = " + (char) st.ttype);



Other = &
Word = copy
Other = ;
Number = 2012.0
Other = ,
Word = smartjava.org

This was an example of how to get a tokenizer from a FileReader in Java.

Do you want to know how to develop your skillset to become a Java Rockstar?

Subscribe to our newsletter to start Rocking right now!

To get you started we give you our best selling eBooks for FREE!


1. JPA Mini Book

2. JVM Troubleshooting Guide

3. JUnit Tutorial for Unit Testing

4. Java Annotations Tutorial

5. Java Interview Questions

6. Spring Interview Questions

7. Android UI Design


and many more ....


Receive Java & Developer job alerts in your Area

I have read and agree to the terms & conditions


Notify of

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Inline Feedbacks
View all comments