Data Center consists of multiple levels of Load Balancing. It starts with Web Tier load balancer which will take the traffic from the internet and pass the traffic to mid tier load balancers. Web Tier load balancers usually are dedicated costly machines brought from the market. It is in the mid tier load balancers, Software Load Balancer(SLB) would fit in. SLB can run Bare Metals, VM or Docker containers and it is open source, so it is cost effective to scale horizontally. All SLB instances will contain all the routing rules and it can be either pushed or pulled from a central repository. Updating the routing rules doesn't affect the existing connection, so frequent updation won't affect the reliability of the system.
Neutrino
Pipeline Architecture
Request and Response Pipeline is an important feature of SLB. It makes the SLB modular and pluggable.
Netty is a NIO client server framework which enables quick and easy development of network applications such as protocol servers and clients. Netty ChannelPipeline architecture. ChannelPipeline implements an advanced form of the Intercepting Filter pattern to give a user full control over how an event is handled and how the ChannelHandlers in a pipeline interact with each other.
Pros
- Proven Pipeline technology
- Extensive documentation (User Guide, How-to, Blogs, Books)
- Fully asynchronous and event-driven
- Handlers can operate on raw bytes (ByteBuf)
- Flexible factory pattern for creating a pipeline
- Built-in support for short circuiting a pipeline handler by specifying a timeout for handler processing in Netty 4.
Adding new handler in PipeLine
Adding a new Handler in the pipe line is pretty simple. Following is sample code which adds a handler which will print all incoming URLs
import com.ebay.neutrino.NeutrinoRequest;
import io.netty.channel.ChannelDuplexHandler;
import io.netty.channel.ChannelHandler;
import io.netty.channel.ChannelHandlerContext;
/**
* Created by blpaul on 11/10/2015.
*/
@ChannelHandler.Sharable
public class NeutrinoSampleHandler extends ChannelDuplexHandler {
@Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
if (msg instanceof NeutrinoRequest) {
NeutrinoRequest request = (NeutrinoRequest) msg;
System.out.println(request.requestUri());
}
ctx.fireChannelRead(msg);
}
}
Initialize the handler in NeutrinoServiceInitializer.scala
val timeout = new ChannelTimeoutHandler(settings.timeouts)
//Initializing the new Handler
val sampleHandler = new NeutrinoSampleHandler();
pipeline.addLast("user-pipeline", user)
//Adding the new Handler
pipeline.addLast("sampleHandler", sampleHandler)
// Final handler will reroute inbound to our downstream, as available
pipeline.addLast("downstream", downstream)
PoolResolvers
PoolResolvers are used to resolve a request to pool. Supported pool resolvers are CNameResolver and L7AddressResolver. CNameResolver resolves the request to a pool based on the Canonical name(cname) while L7AddressResolverresolves the request based cname and url postfix.
Below is a sample code for adding a resolver for 2FA. If the 2FA token is missing in the header, Neutrino will redirect the request to a 2FA server
import com.ebay.neutrino.*;
import com.ebay.neutrino.config.VirtualPool;
import scala.None;
import scala.Option;
import scala.collection.concurrent.TrieMap;
import java.util.List;
import static com.ebay.neutrino.config.Transport$.*;
/**
* Created by blpaul on 11/10/2015.
*/
public class TwoFAResolver implements PoolResolver {
private Integer version = 0;
private NeutrinoPool twofaPool;
@Override
public Option resolve(NeutrinoPools pools, NeutrinoRequest request) {
// Cache the twofaPool
if (version != pools.version()) {
version = pools.version();
NeutrinoPoolId neutrinoId;
// Since Transport is sealed trait, we have access with MODULE in java
neutrinoId = new NeutrinoPoolId("twofa", MODULE$.apply("http"));
twofaPool = pools.pools().get(neutrinoId).get();
}
String token = request.headers().get("TOKEN");
Option neutrinoPoolOption = Option.apply(null);
// Validate the token
Boolean validated = true;
if (validated == false) {
neutrinoPoolOption = Option.apply(twofaPool);
return neutrinoPoolOption;
}
return neutrinoPoolOption;
}
}
Initialize the pool resolver in NeutrinoPoolResolver.scala
case "layerseven" => new L7AddressResolver
case "twofa" => new TwoFAResolver
Select the pool resolver for the configuration
listeners = [
{
# Should try cname and then skip to default
pool-resolver = ["twofa", "cname", "layerseven"],
port = 9080,
port-alias = 80,
protocol = "http"
}
]
.....
.....
pools = [
{ id = "twofa", protocol = "http", ...
Pool Balancers
Pool Balancers are used to manage the traffic between individual VMs in the pool and it can configured for each pool. Currently, there is support for both RoundRobin and LeastConnection Balancers
Below is sample code for adding a Round Robin Pool Balancer
import com.ebay.neutrino.{NeutrinoNode, NeutrinoRequest}
class RoundRobinBalancer extends Balancer {
type Entry = NeutrinoNode
private val members = new BalancerNodes[Entry]()
private var iter = members.available.iterator
// Re(set) the current membership of the load-balancer
override def rebuild(members: Array[NeutrinoNode]) =
this.members.set(members, identity)
// Resolve an endpoint for request processing
def assign(request: NeutrinoRequest): Option[NeutrinoNode] =
if (members.isEmpty)
None
else
// Store the active iterator
members.synchronized {
if (iter.isEmpty) iter = members.available.iterator
if (iter.hasNext) Option(iter.next._1)
else None
}
// Release an endpoint from request processing
def release(request: NeutrinoRequest, node: NeutrinoNode) = {}
}
Initialize the balancer in BalancerSettings.scala
val RoundRobin = BalancerSettings(classOf[RoundRobinBalancer], None)
....
....
def apply(balancer: String): BalancerSettings =
balancer toLowerCase match {
case "rr" | "round-robin" => RoundRobin
Add the new Balancer to the pool
pools = [
{ id = "twofa", protocol = "http", balancer = "round-robin",