Microservices and Reactive Architecture

Micro services

The Microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms. 

Critical characteristics of a Micro Services are

1. Componentization via services
2. Organized around Business Capabilities
3. Smart End points
4. Infrastructure Automation
5. Design for failure

Key considerations for efficient Micro Services development and deployment.

1. Infrastructure Automation and Rapid Provisioning
2. Basic Monitoring
3. Rapid Application Deployment
4. Devops 

Advantages for Microservices are 

1. Ability to deploy different services independently
2. Reliability, due to the ability for a service to run even if another service is down. 
3. Ability to build Services on different platforms, e.g. as a way to experiment with a new language.

Reactive Architecture

The need for error-tolerance and fast reaction in dynamic environments have resulted in reactive architectures. They have task-specific modules that initiate direct reactions in response to specific situations that occur in the environment. 

Key characteristics of Reactive Architecture are: 

1. Responsive
2. Resilient
3. Elastic/Scalable
4. Message Driven

Systems built as Reactive Systems are more flexible, loosely-coupled and scalable. They are significantly more tolerant of failure

Benefits of both Styles:

While Micro services enables looking at application in terms of small individually deployable services, Reactive Architecture helps building applications which can scale as need (at required later by scaling required services) and ensure fail over. Together they bring ease of maintenance, reliability, scalability and fail over to the application. 

Microservice implementation using OSGI, Apache Karaf and CXF

OSGi (Open Service Gateway Initiative) is a Java framework for developing and deploying modular software programs and libraries. It’s a specification that defines how to develop components that can be deployed and invoked as Services (as in SOA)

OSGi only defines specifications for component development and basic execution environment. OSGi specifications are implemented by projects like Apache Felix.

Apache Karaf adds utilities to Apache Felix. These include ability to Hot deploy, Logging and Monitoring etc.
It also includes a Container that contains these Utilities and along with Apache Felix. It also provides ability to Remote login to the container. Services developed using OSGi specs can be deployed into this container.

Karaf container sounds similar to Docker. In case of Docker developer need to build an image specifying OS, app server (like tomcat) and then application can be deployed in that Image. Where as Karaf Container has all necessary environment to run OSGi services. Developer need to bring the container up and deploy the services.

OSGi services when complimented by the features provides by Karaf can be a good choice to implement Microservices architecture. I don't feel OSGi service (deployed on plain Apache Helix) can meet all required criteria to be Microservice architecture. Because features like Scaling, Monitoring etc need to be custom implemented.

In fact OSGi services need to be exposed as REST services (using Apache CXF) so that they can be invoked from any application/platform/technology.

Below are the quick steps to set up and get basic service with REST interface up on Apache Karaf.

Lets first define a simple service

Interface:

public interface MyService {

public String printMessage();
}

Service Implementation

public class MyServiceImpl implements MyService {

@Override
public String printMessage() {
// TODO Auto-generated method stub
return "My Service";

}

}


Next define a REST interface for this service using Jersey framework

@Path("/myService")
public class MyRestService {

@Produces({ MediaType.APPLICATION_JSON })
@GET
public String getMessage() {
MyService service = new MyServiceImpl();
return service.printMessage();
}
}


Create a simple Java project to contain above code components. Compile, build and package as a Jar file. Now lets setup Karaf and CXF to deploy this bundle.

Below are the quick steps

1. Install Karaf container. 
• Download Karaf container(binary distribution for quick start) from http://karaf.apache.org/download.html
• Unzip the contents to
2. Start the container
• /bin/karaf
• Karaf shell should be active now (karaf@root()>)
3. In Karaf shell execute below commands to install Apache CXF and HTTP features to enable http interface and rest services
• feature:repo-add cxf 3.1.5
• feature:install http cxf-jaxws http-whiteboard
• feature:install  cxf-jaxrs
4. Copy the Jar bundle to /deploy
5. Command "list" should list the service just deployed if deployment is successful.
6. Use command "bundle:diag" to check errors with last operation

Use URL "http://localhost:8181/cxf/sample/myService" to test the Service

This post discussed basic service implementation with Karaf. Shall look at other aspects of Microservice implementation in a later post.

Docker on Mac OS 10.10.

Well, Docker is everywhere now!! For last few weeks studying their applicability in DevOps, Micro services etc. Today wanted to make my hands dirty with some Docker samples. But its not smooth start. Just wanted to share the issues and resolution at one place.

After installing docker as per documentation on Mac (OS X 10.10) and launching the CLI first error got was “Default is not running. Please start this with docker-machine start default.” After googling around and analysing different solutions, found the issue is with Virtualbox version. (I had Viirtualbox version 5.0 already installed.). The issue is resolved with installing test Test-Build 5.0 (https://www.virtualbox.org/wiki/Testbuilds). The link for issue is  https://www.virtualbox.org/ticket/14412 

After restarting Docker, next error I got is with ca.pem file. After starting Default VM error shown is “ca.pem: no such file or directory”. After executing below two commands, docker started normally

docker-machine regenerate-certs default

docker-machine restart default

Below are the links that helped me solve the issues

https://github.com/docker/docker/issues/15712

http://stackoverflow.com/questions/32240742/after-installing-docker-docker-machine-env-default-fails-because-ca-pem-is-not

Code on Your Own device (CYOD)

In my blog post in Feb, 2011  "Work on move... or coding on mobile" I envisioned possibility of coding on a mobile device. Today I was talking to my colleague who said he uses only iPad for work no Laptop!!. I asked how do you code? He connects to server using Microsoft Remote Desktop, connects a key board to iPad and codes!!. Some time back from I got to know  from IoT team of a light weight version of eclipse that works on Raspberry PI which got 1GB RAM. Today I use a phone that got 3GB RAM!!!. Don't think its long way to have Eclipse and JRE on Mobile devices. (I guess there are some issues to install eclipse directly now but there are alternatives like Ubuntu Virtual Box on Android and Eclipse on top of it, root the device and install Linux etc).

So can we see "Code on Your Own device" movement soon!!!

Facebook for Business - Mind Map

I tried to summarize my understanding of Facebook's applicability for Business. 

Apart from Facebook's Graph API and Ads API below are few things worth learning about

•Open Graph

–Open Graph lets apps tell stories on Facebook through a structured, strongly typed API.

–Stories used share the things people are doing on webpage/App. Ex: “X has finished reading Y book on Google Reads”, “A bought product B on XYZ Store”. These are posted on user’s Facebook page as posts

–Insights API can be used to track the stories, likes and comments on them.

•Facebook exchange

–Facebook Exchange helps you reach people who have expressed an interest through their online behavior so you can reach them with a similar kind of product or service.

–Ex: Target user who searched for product or clicked a web site.

–Uses Demand-side platforms

•PMD Center

–The Facebook Preferred Marketing Developer (PMD) program is a community of hundreds of best-in-class developers /companies Facebook marketing solutions.

–Companies get additional support from Facebook platform representatives

•Public Content Solutions (PCS)

–Partnership with companies which  are experts in integrating Facebook and Instagram into their customers’ experiences connecting audiences and fans

–Privileged access to data and content APIs and dedicated support resources

–Direct collaboration with Facebook business development to help support and grow your business

Issue with Facebook Graph API

Facing an issue with Facebook API. Looking out for help to understand the reasons and resolve the same
We are trying get public wall messages containing a particular keyword.We created application with permissions including read_stream and manage_posts. Application visibility is public and most of other settings are defaults. We could see API working and getting the feeds. Last week we observed an issue. Search API is not picking up messages from certain users. We picked up couple of such ids did some testing. Privacy settings all seems to be fine. Posts visibility settings are public both at feed level and account leven. Platform is turned on. Visibility to search engines is on. For these selected IDs we tried to get wall feeds using their IDs. Below are the observations.

1. For a certain ID we could get all messages both in search and wall retrieval. Both with our application key and default Graph API Explorer access key.
2. For one ID we for messages using Graph API Explorer but not with our application. With our application only feeds we see response are friend request accepts and likes to some other public posts.
3. For another ID Graph API explorer is giving empty results and out application key is giving likes etc as observed in above case.
4. For one of the Use we asked him to turn off the Platform, we were getting messages before. When turned off we stopped getting messages. When turned on we didn't get all messages but only likes etc.
5. When we use Social Mention (socialmention.com) and search for keyword, we could see all messages.

We feel it has to do something with the Application and access key permissions but couldn't get much details. Any help in cracking this is welcome.

Update (May 05, 2014):
The issue is with the way access token generated.

While generating access key we can select permissions to match application requirements. Any thing beyond basic permissions require permission from User. For example if we choose user_location permission, user need to provide permission to get that information. If no additional permissions selected, application will not require any permission and can access all the public data. But if any additional permissions are selected and user not given permission, we cannot get even the public data of that user. With default permissions I generated access key and could get all public data as expected.

"Apache Flume: Distributed Log Collection for Hadoop" - Book review

We have been working on Apache Flume for quite some time now. We used it to load data from Social network into MongoDB and also for log collection. Recently I read a book on Apache Flume titled "Apache Flume: Distributed Log Collection for Hadoop".

 

This is a good starter material for a serious Flume developer. 2nd Chapter provides a good step by step guide to setup and get running with Flume. Liked the way the flow is presented and the mention about important arguments that can be provided to Flume command line. Other useful information from this book is about the Monitoring tools. Though the description is brief good to have introduction to some tools. Overall book gives good details with examples on Flume flow and architecture which includes Channels, agents, sinks, interceptors etc. It would have been more helpful if last two chapters were elaborated more.