Playing with Thrift and Java

Thrift is an interface definition language that is used to define and create services for numerous languages, including Java. It is used as a remote procedure call (RPC) framework and was developed at Facebook.

At Intelliseq we are using thrift as a communication protocol for our genequery noSQL database. In this post I will describe basic thrift server and a test client.

First, we need to download and install thrift. See http://thrift.apache.org/.

Next we need to design our interface. This is what I like about thrift. You need to design interface first. It is my favorite approach to software architecture design. Below is a simple interface of service with enum, structure, exception and few methods. I didn't include collections, imports, one way methods and few more less important featuresof thrift. You can read more about them in this great missing guide.

namespace java pl.intelliseq.largedata.thrift

struct Message {
 1: required string message
}

struct User {
 1: required string firstname
 2: required string lastname
}

enum Wrong {
 FIRST = 1,
 SECOND = 2
}

exception InvalidFirst {
 1: string why
}

ExampleService {
 bool isAlive(),
 Message getHello(1:User user),
 void getError(1: Wrong wrong) throws (1:InvalidFirst ouch)
}

We can autogenerate code with this command. Assuming that our thrift file is in thrift subdirectory.

thrift --gen java thrift/Exampleservice.thrift

All classes were autogenerated and placed in gen-java directory. We can set this directory as src directory. Next, we will implement all methods:

package pl.intelliseq.largedata.thrift;

import org.apache.thrift.TException;

public class ThriftHandler implements ExampleService.Iface {

 @Override
 public boolean isAlive() throws TException {
  return true;
 }

 @Override
 public Message getHello(User user) throws TException {
  Message message = new Message();
  message.setMessage("Hello " + user.getFirstname() + " " + user.getLastname());
  return message;
 }

 @Override
 public void getError(Wrong wrong) throws InvalidFirst, TException {
  if (wrong.equals(Wrong.FIRST)) throw new InvalidFirst();
 }

}

We have also to set up our server. We need to run it in separate thread if we want to test it.

package pl.intelliseq.largedata.thrift;

import org.apache.thrift.protocol.TBinaryProtocol;
import org.apache.thrift.server.TServer;
import org.apache.thrift.server.TThreadedSelectorServer;
import org.apache.thrift.transport.TFramedTransport;
import org.apache.thrift.transport.TNonblockingServerSocket;
import org.apache.thrift.transport.TNonblockingServerTransport;
import org.apache.thrift.transport.TTransportException;

public class ThriftServer implements Runnable {

 TServer server;
 
 public void init() throws InterruptedException, TTransportException {
  System.out.println("Starting server on port 9090 ...");
  ThriftHandler handler = new ThriftHandler();
  ExampleService.Processor processor = new ExampleService.Processor(
    handler);
  TNonblockingServerTransport trans = new TNonblockingServerSocket(9090);
  TThreadedSelectorServer.Args args = new TThreadedSelectorServer.Args(trans);
  args.transportFactory(new TFramedTransport.Factory());
  args.protocolFactory(new TBinaryProtocol.Factory());
  args.processor(processor);
  args.selectorThreads(4);
  args.workerThreads(32);
  server = new TThreadedSelectorServer(args);

  new Thread(this).start();

  while(!server.isServing()) {Thread.sleep(1); };
  System.out.println("Serving on port 9090 ...");
 }
 
 public void run() {
  server.serve();
 }
}

And finally, we will write our test:

package pl.intelliseq.largedata.thrift;

import static org.junit.Assert.*;

import org.apache.thrift.TException;
import org.apache.thrift.protocol.TBinaryProtocol;
import org.apache.thrift.protocol.TProtocol;
import org.apache.thrift.transport.TFramedTransport;
import org.apache.thrift.transport.TSocket;
import org.apache.thrift.transport.TTransport;
import org.apache.thrift.transport.TTransportException;
import org.junit.After;
import org.junit.Before;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.ExpectedException;
import org.junit.runner.RunWith;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration({"file:conf/thrift-conf.xml"})
public class ThriftServerTest {

 TTransport transport;
 ExampleService.Client client;
 
 @Before
 public void init() throws TTransportException {
  transport = new TFramedTransport(new TSocket("localhost", 9090));
  transport.open();
  TProtocol protocol = new TBinaryProtocol(transport);
  client = new ExampleService.Client(protocol);
 }
 
 @After
 public void destroy() {
  transport.close();
 }

 @Rule
 public ExpectedException exception = ExpectedException.none();
 
 @Test
 public void isAliveTest() throws TException {
  assertTrue(client.isAlive());
 }
 
 @Test
 public void getHelloTest() throws TException {
  User user = new User();
  user.setFirstname("John");
  user.setLastname("Doe");
  assertEquals(client.getHello(user).getMessage(), "Hello John Doe");
 }
 
 @Test
 public void getErrorTest() throws TException {
  exception.expect(InvalidFirst.class);
  Wrong wrong = Wrong.FIRST;
  client.getError(wrong);
 }
 
 @Test
 public void getErrorSecondTest() throws TException {
  Wrong wrong = Wrong.SECOND;
  client.getError(wrong);
 }

}

That's it. Works like a charm. You can clone working project here: https://github.com/marpiech/largedatablog.git -b thrift thrift-and-java

Problems:

How to get rid of ugly warnings caused by autogenerated thrift code? (in Eclipse)
Create new source directory: e.g. src/thrift. Copy autogenerated code to the new directory. Then, Right click on the directory -> Build Path -> Use as Source Folder. Right click again -> Build Path -> Configure Build Path... -> Ignore optional compile problems -> Toggle.

org.apache.thrift.TApplicationException: [your method] failed: unknown result
your server implementation returned null or see: http://stackoverflow.com/questions/4244350/how-thread-safe-is-thrift-re-i-seem-to-have-requests-disrupting-one-another

TNonblockingServer.java [line number] Read an invalid frame size of [integer number]. Are you using TFramedTransport on the client side?
Wrap your TSocket into TFramedTransport (i.e. new TFramedTransport(new TSocket("localhost", 9090)))