Data Center consists of multiple levels of Load Balancing. It starts with Web Tier load balancer which will take the traffic from the internet and pass the traffic to mid tier load balancers. Web Tier load balancers usually are dedicated costly machines brought from the market. It is in the mid tier load balancers, Software Load Balancer(SLB) would fit in. SLB can run Bare Metals, VM or Docker containers and it is open source, so it is cost effective to scale horizontally. All SLB instances will contain all the routing rules and it can be either pushed or pulled from a central repository. Updating the routing rules doesn't affect the existing connection, so frequent updation won't affect the reliability of the system.

Pipeline Architecture

Request and Response Pipeline is an important feature of SLB. It makes the SLB modular and pluggable.

Netty is a NIO client server framework which enables quick and easy development of network applications such as protocol servers and clients. Netty ChannelPipeline architecture. ChannelPipeline implements an advanced form of the Intercepting Filter pattern to give a user full control over how an event is handled and how the ChannelHandlers in a pipeline interact with each other.


  • Proven Pipeline technology
  • Extensive documentation (User Guide, How-to, Blogs, Books)
  • Fully asynchronous and event-driven
  • Handlers can operate on raw bytes (ByteBuf)
  • Flexible factory pattern for creating a pipeline
  • Built-in support for short circuiting a pipeline handler by specifying a timeout for handler processing in Netty 4.

Adding new handler in PipeLine

Adding a new Handler in the pipe line is pretty simple. Following is sample code which adds a handler which will print all incoming URLs

import com.ebay.neutrino.NeutrinoRequest;

 * Created by blpaul on 11/10/2015.
public class NeutrinoSampleHandler extends ChannelDuplexHandler {

	public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
		if (msg instanceof NeutrinoRequest) {
			NeutrinoRequest request = (NeutrinoRequest) msg;



Initialize the handler in NeutrinoServiceInitializer.scala

val timeout = new ChannelTimeoutHandler(settings.timeouts)
//Initializing the new Handler
val sampleHandler = new NeutrinoSampleHandler();

pipeline.addLast("user-pipeline", user)

//Adding the new Handler
pipeline.addLast("sampleHandler", sampleHandler)

// Final handler will reroute inbound to our downstream, as available
pipeline.addLast("downstream", downstream)


PoolResolvers are used to resolve a request to pool. Supported pool resolvers are CNameResolver and L7AddressResolver. CNameResolver resolves the request to a pool based on the Canonical name(cname) while L7AddressResolverresolves the request based cname and url postfix.

Below is a sample code for adding a resolver for 2FA. If the 2FA token is missing in the header, Neutrino will redirect the request to a 2FA server

import com.ebay.neutrino.*;

import com.ebay.neutrino.config.VirtualPool;
import scala.None;
import scala.Option;
import scala.collection.concurrent.TrieMap;

import java.util.List;

import static com.ebay.neutrino.config.Transport$.*;

 * Created by blpaul on 11/10/2015.

public class TwoFAResolver implements PoolResolver {

    private Integer version = 0;

    private NeutrinoPool twofaPool;

    public Option resolve(NeutrinoPools pools, NeutrinoRequest request) {

		// Cache the twofaPool 
        if (version != pools.version()) {
            version = pools.version();
            NeutrinoPoolId neutrinoId;
            // Since Transport is sealed trait, we have access with MODULE in java
            neutrinoId = new NeutrinoPoolId("twofa", MODULE$.apply("http"));
            twofaPool  = pools.pools().get(neutrinoId).get();

        String token = request.headers().get("TOKEN");

        Option neutrinoPoolOption = Option.apply(null);
		// Validate the token
        Boolean validated = true;
        if (validated == false) {
            neutrinoPoolOption = Option.apply(twofaPool);
            return neutrinoPoolOption;

        return neutrinoPoolOption;


Initialize the pool resolver in NeutrinoPoolResolver.scala

 case "layerseven"   => new L7AddressResolver
 case "twofa"   => new TwoFAResolver

Select the pool resolver for the configuration

listeners = [
      # Should try cname and then skip to default
      pool-resolver = ["twofa", "cname", "layerseven"],
      port = 9080,
      port-alias = 80,
      protocol = "http"
    pools = [
    { id = "twofa", protocol = "http", ...

Pool Balancers

Pool Balancers are used to manage the traffic between individual VMs in the pool and it can configured for each pool. Currently, there is support for both RoundRobin and LeastConnection Balancers

Below is sample code for adding a Round Robin Pool Balancer

import com.ebay.neutrino.{NeutrinoNode, NeutrinoRequest}

class RoundRobinBalancer extends Balancer {

  type Entry = NeutrinoNode

  private val members = new BalancerNodes[Entry]()
  private var iter = members.available.iterator

  // Re(set) the current membership of the load-balancer
  override def rebuild(members: Array[NeutrinoNode]) =
    this.members.set(members, identity)

  // Resolve an endpoint for request processing
  def assign(request: NeutrinoRequest): Option[NeutrinoNode] =
    if (members.isEmpty)
      // Store the active iterator
      members.synchronized {
        if (iter.isEmpty) iter = members.available.iterator
        if (iter.hasNext) Option(
        else None

  // Release an endpoint from request processing
  def release(request: NeutrinoRequest, node: NeutrinoNode) = {}


Initialize the balancer in BalancerSettings.scala

val RoundRobin = BalancerSettings(classOf[RoundRobinBalancer], None)
 def apply(balancer: String): BalancerSettings =
    balancer toLowerCase match {
      case "rr" | "round-robin" => RoundRobin

Add the new Balancer to the pool

pools = [
  { id = "twofa", protocol = "http", balancer = "round-robin",