This article is a tutorial on demonstrating the usage of the Java Nio ByteBuffer. All examples are done in the form of unit tests to easily prove the expectations of the API.
1. Introduction
The ByteBuffer class is an abstract class which also happens to extend Buffer and implement Comparable. A Buffer is simply a linear finite sized container for data of a certain primitive type. It exhibits the following properties:
- capacity: the number of elements it contains
- limit : the index of where the data it contains ends
- position : the next element to be read or written
ByteBuffer has these properties but also displays a host of semantic properties of it’s own. According to the ByteBuffer API the abstraction defines six categories of operations. They are:
get(...)andput(...)operations that operate relatively (in terms of the current position) and absolutely (by supplying an index)- bulk
get(...)operation done relatively (in terms of the current position) which will get a number of bytes from the ByteBuffer and place it into the argumentarraysupplied to theget(...)operation - bulk
put(...)operation done absolutely by supplying anindexand the content to be inserted - absolute and relative
get(...)andput(...)operations that get and put data of a specific primitive type, making it convenient to work with a specific primitive type when interacting with the ByteBuffer - creating a “view buffer’ or view into the underlying ByteBuffer by proxying the underlying data with a Buffer of a specific primitive type
- compacting, duplicating and slicing a ByteBuffer
A ByteBuffer is implemented by the HeapByteBuffer and MappedByteBuffer abstractions. HeapByteBuffer further specializes into HeapByteBufferR (R being read-only), which will very conveniently throw a ReadOnlyBufferException and should you try to mutate it via it’s API. The MappedByteBuffer is an abstract class which is implemented by DirectByteBuffer. All of theHeapByteBuffer implementations are allocated on the heap (obviously) and thus managed by the JVM.
2. Technologies used
The example code in this article was built and run using:
- Java 1.8.101 (1.8.x will do fine)
- Maven 3.3.9 (3.3.x will do fine)
- Spring source tool suite 4.6.3 (Any Java IDE would work)
- Ubuntu 16.04 (Windows, Mac or Linux will do fine)
3. Overview
A ByteBuffer is created via the the two static factory methods:
allocate(int)this will allocate aHeapByteBufferwith the capacity specified by theintargumentallocateDirect(int)this will allocate aDirectByteBufferwith the capacity specified by theintargument
The ByteBuffer class affords us the luxury of a fluent interface through much of it’s API, meaning most operations will return a ByteBuffer result. This way we can obtain a ByteBuffer by also wrapping a byte [], slicing a piece of another ByteBuffer, duplicating an existing ByteBuffer and performing get(...)and put(...)operations against an existing ByteBuffer. I encourage you to review the ByteBuffer API to understand the semantics of it’s API.
So why the distinction between direct and non-direct? It comes down to allowing the Operating System to access memory addresses contiguously for IO operations (hence being able to shove and extract data directly from the memory address) as opposed to leveraging the indirection imposed by the abstractions in the JVM for potentially non-contiguous memory spaces. Because the JVM cannot guarantee contiguous memory locations for HeapByteBuffer allocations the Operating System cannot natively shove and extract data into these types of ByteBuffers. So generally the rule of thumb is should you be doing a lot of IO, then the best approach is to allocate directly and re-use the ByteBuffer. Be warned DirectByteBuffer instances are not subject to the GC.
4. Test cases
To ensure determinism we have been explicit about the Charset in use, therefore any encoding of bytes or decoding of bytes will use the explicit UTF-16BE Charset.
Relative Get and Put operations Test cases
public class RelativeGetPutTest extends AbstractTest {
@Test
public void get() {
final ByteBuffer buffer = ByteBuffer.wrap("Hello world!".getBytes(BIG_ENDIAN_CHARSET));
final byte a = buffer.get();
final byte b = buffer.get();
assertEquals("Buffer position invalid", 2, buffer.position());
assertEquals("'H' not the first 2 bytes read", "H", new String(new byte[] { a, b }, BIG_ENDIAN_CHARSET));
}
@Test
public void put() {
final ByteBuffer buffer = ByteBuffer.allocate(24);
buffer.put("H".getBytes(BIG_ENDIAN_CHARSET));
buffer.put("e".getBytes(BIG_ENDIAN_CHARSET));
buffer.put("l".getBytes(BIG_ENDIAN_CHARSET));
buffer.put("l".getBytes(BIG_ENDIAN_CHARSET));
buffer.put("o".getBytes(BIG_ENDIAN_CHARSET));
buffer.put(" ".getBytes(BIG_ENDIAN_CHARSET));
buffer.put("e".getBytes(BIG_ENDIAN_CHARSET));
buffer.put("a".getBytes(BIG_ENDIAN_CHARSET));
buffer.put("r".getBytes(BIG_ENDIAN_CHARSET));
buffer.put("t".getBytes(BIG_ENDIAN_CHARSET));
buffer.put("h".getBytes(BIG_ENDIAN_CHARSET));
buffer.put("!".getBytes(BIG_ENDIAN_CHARSET));
assertEquals("Buffer position invalid", 24, buffer.position());
buffer.flip();
assertEquals("Text data invalid", "Hello earth!", byteBufferToString(buffer));
}
@Test
public void bulkGet() {
final ByteBuffer buffer = ByteBuffer.wrap("Hello world!".getBytes(BIG_ENDIAN_CHARSET));
final byte[] output = new byte[10];
buffer.get(output);
assertEquals("Invalid bulk get data", "Hello", new String(output, BIG_ENDIAN_CHARSET));
assertEquals("Buffer position invalid", 10, buffer.position());
}
@Test
public void bulkPut() {
final ByteBuffer buffer = ByteBuffer.wrap("Hello world!".getBytes(BIG_ENDIAN_CHARSET));
final byte[] output = new String("earth.").getBytes(BIG_ENDIAN_CHARSET);
buffer.position(12);
buffer.put(output);
assertEquals("Buffer position invalid", 24, buffer.position());
buffer.flip();
assertEquals("Text data invalid", "Hello earth.", byteBufferToString(buffer));
}
@Test
public void getChar() {
final ByteBuffer buffer = ByteBuffer.wrap("Hello world!".getBytes(BIG_ENDIAN_CHARSET));
buffer.mark();
final byte a = buffer.get();
final byte b = buffer.get();
buffer.reset();
char value = buffer.getChar();
assertEquals("Buffer position invalid", 2, buffer.position());
assertEquals("'H' not the first 2 bytes read", "H", new String(new byte[] { a, b }, BIG_ENDIAN_CHARSET));
assertEquals("Value and byte array not equal", Character.toString(value), new String(new byte[] { a, b }, BIG_ENDIAN_CHARSET));
}
@Test
public void putChar() {
final ByteBuffer buffer = ByteBuffer.wrap("Hello world!".getBytes(BIG_ENDIAN_CHARSET));
buffer.position(22);
buffer.putChar('.');
assertEquals("Buffer position invalid", 24, buffer.position());
buffer.flip();
assertEquals("Text data invalid", "Hello world.", byteBufferToString(buffer));
}
}
The above suite of test cases demonstrate relative get()and put()operations. These have a direct effect on certain ByteBuffer attributes (position and data). In addition to being able to invoke these operations with byte arguments or receive byte arguments we also demonstrate usage of the putChar()and getChar(...)methods which conveniently act on the matching primitive type in question. Please consult the API for more of these convenience methods
Absolute Get and Put operations Test cases
public class AbsoluteGetPutTest extends AbstractTest {
@Test
public void get() {
final ByteBuffer buffer = ByteBuffer.wrap("Hello world!".getBytes(BIG_ENDIAN_CHARSET));
final byte a = buffer.get(0);
final byte b = buffer.get(1);
assertEquals("Buffer position invalid", 0, buffer.position());
assertEquals("'H' not the first 2 bytes read", "H", new String(new byte[] { a, b }, BIG_ENDIAN_CHARSET));
}
@Test
public void put() {
final ByteBuffer buffer = ByteBuffer.wrap("Hello world!".getBytes(BIG_ENDIAN_CHARSET));
final byte[] period = ".".getBytes(BIG_ENDIAN_CHARSET);
int idx = 22;
for (byte elem : period) {
buffer.put(idx++, elem);
}
assertEquals("Position must remian 0", 0, buffer.position());
assertEquals("Text data invalid", "Hello world.", byteBufferToString(buffer));
}
@Test
public void getChar() {
final ByteBuffer buffer = ByteBuffer.wrap("Hello world!".getBytes(BIG_ENDIAN_CHARSET));
char value = buffer.getChar(22);
assertEquals("Buffer position invalid", 0, buffer.position());
assertEquals("Invalid final character", "!", Character.toString(value));
}
@Test
public void putChar() {
final ByteBuffer buffer = ByteBuffer.wrap("Hello world!".getBytes(BIG_ENDIAN_CHARSET));
buffer.putChar(22, '.');
assertEquals("Buffer position invalid", 0, buffer.position());
assertEquals("Text data invalid", "Hello world.", byteBufferToString(buffer));
}
}
The above suite of test cases demonstrate usage of the absolute variants of the get(...)and put(...)operations. Interestingly enough, only the underlying data is effected (put(...)) as the position cursor is not mutated owing to the method signatures providing client code the ability to provide an index for the relevant operation. Again convenience methods which deal with the various primitive types are also provided and we demonstrate use of the ...Char(...)variants thereof.
ViewBuffer Test cases
public class ViewBufferTest extends AbstractTest {
@Test
public void asCharacterBuffer() {
final ByteBuffer buffer = ByteBuffer.wrap("Hello world!".getBytes(BIG_ENDIAN_CHARSET));
final CharBuffer charBuffer = buffer.asCharBuffer();
assertEquals("Buffer position invalid", 0, buffer.position());
assertEquals("CharBuffer position invalid", 0, charBuffer.position());
assertEquals("Text data invalid", charBuffer.toString(), byteBufferToString(buffer));
}
@Test
public void asCharacterBufferSharedData() {
final ByteBuffer buffer = ByteBuffer.wrap("Hello world!".getBytes(BIG_ENDIAN_CHARSET));
final CharBuffer charBuffer = buffer.asCharBuffer();
assertEquals("Buffer position invalid", 0, buffer.position());
assertEquals("CharBuffer position invalid", 0, charBuffer.position());
final byte[] period = ".".getBytes(BIG_ENDIAN_CHARSET);
int idx = 22;
for (byte elem : period) {
buffer.put(idx++, elem);
}
assertEquals("Text data invalid", "Hello world.", byteBufferToString(buffer));
assertEquals("Text data invalid", charBuffer.toString(), byteBufferToString(buffer));
}
}
In addition to the various convenience get(...)and put(...)methods that deal with the various primitive types ByteBuffer provides us with an assortment of methods that provide primitive ByteBuffer views of the underlying data eg: asCharBuffer()demonstrates exposing a Character Buffer view of the underlying data.
Miscellaneous ByteBuffer Test cases
public class MiscBufferTest extends AbstractTest {
@Test
public void compact() {
final ByteBuffer buffer = ByteBuffer.allocate(24);
buffer.putChar('H');
buffer.putChar('e');
buffer.putChar('l');
buffer.putChar('l');
buffer.putChar('o');
buffer.flip();
buffer.position(4);
buffer.compact();
assertEquals("Buffer position invalid", 6, buffer.position());
buffer.putChar('n');
buffer.putChar('g');
assertEquals("Buffer position invalid", 10, buffer.position());
buffer.flip();
assertEquals("Invalid text", "llong", byteBufferToString(buffer));
}
@Test
public void testDuplicate() {
final ByteBuffer buffer = ByteBuffer.wrap("Hello world!".getBytes(BIG_ENDIAN_CHARSET));
final ByteBuffer duplicate = buffer.duplicate();
assertEquals("Invalid position", 0, duplicate.position());
assertEquals("Invalid limit", buffer.limit(), duplicate.limit());
assertEquals("Invalid capacity", buffer.capacity(), duplicate.capacity());
buffer.putChar(22, '.');
assertEquals("Text data invalid", "Hello world.", byteBufferToString(buffer));
assertEquals("Text data invalid", byteBufferToString(duplicate), byteBufferToString(buffer));
}
@Test
public void slice() {
final ByteBuffer buffer = ByteBuffer.wrap("Hello world!".getBytes(BIG_ENDIAN_CHARSET));
buffer.position(12);
final ByteBuffer sliced = buffer.slice();
assertEquals("Text data invalid", "world!", byteBufferToString(sliced));
assertEquals("Invalid position", 0, sliced.position());
assertEquals("Invalid limit", buffer.remaining(), sliced.limit());
assertEquals("Invalid capacity", buffer.remaining(), sliced.capacity());
buffer.putChar(22, '.');
assertEquals("Text data invalid", "world.", byteBufferToString(sliced));
}
@Test
public void rewind() {
final ByteBuffer buffer = ByteBuffer.wrap("Hello world!".getBytes(BIG_ENDIAN_CHARSET));
final byte a = buffer.get();
final byte b = buffer.get();
assertEquals("Invalid position", 2, buffer.position());
buffer.rewind();
assertEquals("Invalid position", 0, buffer.position());
assertSame("byte a not same", a, buffer.get());
assertSame("byte a not same", b, buffer.get());
}
@Test
public void compare() {
final ByteBuffer a = ByteBuffer.wrap("Hello world!".getBytes(BIG_ENDIAN_CHARSET));
final ByteBuffer b = ByteBuffer.wrap("Hello world!".getBytes(BIG_ENDIAN_CHARSET));
assertTrue("a is not the same as b", a.compareTo(b) == 0);
}
@Test
public void compareDiffPositions() {
final ByteBuffer a = ByteBuffer.wrap("Hello world!".getBytes(BIG_ENDIAN_CHARSET));
final ByteBuffer b = ByteBuffer.wrap("Hello world!".getBytes(BIG_ENDIAN_CHARSET));
a.position(2);
assertTrue("a is the same as b", a.compareTo(b) != 0);
}
}
5. Summary
In this tutorial we learned a bit about ByteBuffers, we understood how to create one, the various types, why we have the different types and when to use them as well as the core semantic operations defined by the abstraction.
ByteBuffers are not thread safe and hence many operations on it, need to be guarded against to ensure that multiple threads do not corrupt the data or views thereon. Be wary of relative get(...) and put(...) operations as these do sneaky things like advancing the ByteBuffers position.
Wrapping, slicing and duplicating all point to the byte [] they wrapped or the ByteBuffer they sliced / duplicated. Changes to the source input or the resulting ByteBuffers will effect each other. Luckily with slice(...)and duplicate(...)the position, mark and limit cursors are independent.
When toggling between reading data into a ByteBuffer and writing the contents from that same ByteBuffer it is important to flip()the ByteBuffer to ensure the limit is set to the current position, the current position is reset back to 0 and the mark, if defined, is discarded. This will ensure the ensuing write will be able to write what was just read. Partial writes in this context can be guarded against by calling compact()right before the next iteration of read and is very elegantly demonstrated in the API under compact.
When comparing ByteBuffers the positions matter, ie: you can have segments of a ByteBuffer that are identical and these compare favorably should the two ByteBuffers, in question, have the same position and limit (bytesRemaining()) during comparison.
For frequent high volume IO operations a DirectByteBuffer should yield better results and thus should be preferred.
Converting a byte []into a ByteBuffer can be accomplished by wrapping the byte []via the wrap(...)method. Converting back to a byte []is not always that straight forward. Using the convenient array() method on ByteBuffer only works if the ByteBuffer is backed by a byte []. This can be confirmed via the hasArray()method. A bulk get(...) into an applicably sized byte [] is your safest bet, but be on the guard for sneaky side effects, ie: bumping the positioncursor.
6. Download the source code
This was a Java Nio ByteBuffer tutorial
You can download the full source code of this example here: Java Nio ByteBuffer tutorial
