security

Calculate the CRC Sum of a file

With this example we are going to demonstrate how to calculate the CRC Sum of a file. The Cyclic Redundancy Check is a good way to detect errors and changes to data. In short, in order to calculate the CRC-32 Sum of a file we implemented four different methods :

  • The checksumInputStream(String filepath) method creates a FileInputStream by opening a connection to the file.
  • The checksumBufferedInputStream(String filepath) method creates a BufferedInputStream to save the input stream.
  • The checksumRandomAccessFile(String filepath) method creates a RandomAccessFile to read from the file. The mode argument is set for reading only.
  • The checksumMappedFile(String filepath) method uses a MappedByteBuffer to map the file directly into memory in a Read-only mode, through its FileChannel.
  • In all methods a new CRC32 Object is created. The CRC32 Object is updated for every byte read from each file. The methods are invoked in a main() method and return the CRC32 value of the same file.

Note that for each method implementation time is also calculated, as shown in the output.
 
Let’s take a look at the code snippets and the result output that follow:

package com.javacodegeeks.snippets.core;

import java.io.BufferedInputStream;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.RandomAccessFile;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
import java.util.zip.CRC32;

public class Main {

    public static long checksumInputStream(String filepath) throws IOException {

  InputStream inputStreamn = new FileInputStream(filepath);

  CRC32 crc = new CRC32();


  int cnt;

  

  while ((cnt = inputStreamn.read()) != -1) {


crc.update(cnt);

  }

  return crc.getValue();
    }

    public static long checksumBufferedInputStream(String filepath) throws IOException {

  InputStream inputStream = new BufferedInputStream(new FileInputStream(filepath));

  

  CRC32 crc = new CRC32();


  int cnt;

  

  while ((cnt = inputStream.read()) != -1) {


crc.update(cnt);

  }

  return crc.getValue();
    }

    public static long checksumRandomAccessFile(String filepath) throws IOException {

  RandomAccessFile randAccfile = new RandomAccessFile(filepath, "r");

  long length = randAccfile.length();

  CRC32 crc = new CRC32();


  for (long i = 0; i < length; i++) {


randAccfile.seek(i);


int cnt = randAccfile.readByte();


crc.update(cnt);

  }

  return crc.getValue();
    }

    public static long checksumMappedFile(String filepath) throws IOException {

  FileInputStream inputStream = new FileInputStream(filepath);

  FileChannel fileChannel = inputStream.getChannel();


  int len = (int) fileChannel.size();


  MappedByteBuffer buffer = fileChannel.map(FileChannel.MapMode.READ_ONLY, 0, len);


  CRC32 crc = new CRC32();


  for (int cnt = 0; cnt < len; cnt++) {





int i = buffer.get(cnt);


crc.update(i);

  }

  return crc.getValue();
    }

    public static void main(String[] args) throws IOException {


  String filepath = "C:/Users/nikos7/Desktop/output.txt";


  System.out.println("Input Strea method:");


  long start_timer = System.currentTimeMillis();

  long crc = checksumInputStream(filepath);

  long end_timer = System.currentTimeMillis();


  System.out.println(Long.toHexString(crc));

  System.out.println((end_timer - start_timer) + " ms");

  System.out.println("///////////////////////////////////////////////////////////");


  System.out.println("Buffered Input Stream method:");

  start_timer = System.currentTimeMillis();

  crc = checksumBufferedInputStream(filepath);

  end_timer = System.currentTimeMillis();

  System.out.println(Long.toHexString(crc));

  System.out.println((end_timer - start_timer) + " ms");


  System.out.println("///////////////////////////////////////////////////////////");


  System.out.println("Random Access File method:");

  start_timer = System.currentTimeMillis();

  crc = checksumRandomAccessFile(filepath);

  end_timer = System.currentTimeMillis();

  System.out.println(Long.toHexString(crc));

  System.out.println((end_timer - start_timer) + " ms");


  System.out.println("///////////////////////////////////////////////////////////");


  System.out.println("Mapped File method:");

  start_timer = System.currentTimeMillis();

  crc = checksumMappedFile(filepath);

  end_timer = System.currentTimeMillis();

  System.out.println(Long.toHexString(crc));

  System.out.println((end_timer - start_timer) + " ms");
    }
}

Output:

Input Strea method:
94ccd63e
75 ms
///////////////////////////////////////////////////////////
Buffered Input Stream method:
94ccd63e
5 ms
///////////////////////////////////////////////////////////
Random Access File method:
94ccd63e
99 ms
///////////////////////////////////////////////////////////
Mapped File method:
94ccd63e
8 ms

 
This was an example of calculating the CRC Sum of a file in Java.

Byron Kiourtzoglou

Byron is a master software engineer working in the IT and Telecom domains. He is an applications developer in a wide variety of applications/services. He is currently acting as the team leader and technical architect for a proprietary service creation and integration platform for both the IT and Telecom industries in addition to a in-house big data real-time analytics solution. He is always fascinated by SOA, middleware services and mobile development. Byron is co-founder and Executive Editor at Java Code Geeks.
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

2 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Jentec
Jentec
6 years ago

on big files like 1GB mapped is the fasted way
thank you for post

///////////////////////////////////////////////////////////
Buffered Input Stream method:
3ef92509
29903 ms
///////////////////////////////////////////////////////////
Mapped File method:
3ef92509
2949 ms

mohamed
mohamed
1 year ago
seems that the file is still locked after process, i added inputstream.close(), after ChecksumBufferedInputStream
Back to top button