Code on Your Own device (CYOD)

In my blog post in Feb, 2011  "Work on move... or coding on mobile" I envisioned possibility of coding on a mobile device. Today I was talking to my colleague who said he uses only iPad for work no Laptop!!. I asked how do you code? He connects to server using Microsoft Remote Desktop, connects a key board to iPad and codes!!. Some time back from I got to know  from IoT team of a light weight version of eclipse that works on Raspberry PI which got 1GB RAM. Today I use a phone that got 3GB RAM!!!. Don't think its long way to have Eclipse and JRE on Mobile devices. (I guess there are some issues to install eclipse directly now but there are alternatives like Ubuntu Virtual Box on Android and Eclipse on top of it, root the device and install Linux etc).

So can we see "Code on Your Own device" movement soon!!!

Facebook for Business - Mind Map

I tried to summarize my understanding of Facebook's applicability for Business. 

Apart from Facebook's Graph API and Ads API below are few things worth learning about

•Open Graph

–Open Graph lets apps tell stories on Facebook through a structured, strongly typed API.

–Stories used share the things people are doing on webpage/App. Ex: “X has finished reading Y book on Google Reads”, “A bought product B on XYZ Store”. These are posted on user’s Facebook page as posts

–Insights API can be used to track the stories, likes and comments on them.

•Facebook exchange

–Facebook Exchange helps you reach people who have expressed an interest through their online behavior so you can reach them with a similar kind of product or service.

–Ex: Target user who searched for product or clicked a web site.

–Uses Demand-side platforms

•PMD Center

–The Facebook Preferred Marketing Developer (PMD) program is a community of hundreds of best-in-class developers /companies Facebook marketing solutions.

–Companies get additional support from Facebook platform representatives

•Public Content Solutions (PCS)

–Partnership with companies which  are experts in integrating Facebook and Instagram into their customers’ experiences connecting audiences and fans

–Privileged access to data and content APIs and dedicated support resources

–Direct collaboration with Facebook business development to help support and grow your business

Issue with Facebook Graph API

Facing an issue with Facebook API. Looking out for help to understand the reasons and resolve the same
We are trying get public wall messages containing a particular keyword.We created application with permissions including read_stream and manage_posts. Application visibility is public and most of other settings are defaults. We could see API working and getting the feeds. Last week we observed an issue. Search API is not picking up messages from certain users. We picked up couple of such ids did some testing. Privacy settings all seems to be fine. Posts visibility settings are public both at feed level and account leven. Platform is turned on. Visibility to search engines is on. For these selected IDs we tried to get wall feeds using their IDs. Below are the observations.

1. For a certain ID we could get all messages both in search and wall retrieval. Both with our application key and default Graph API Explorer access key.
2. For one ID we for messages using Graph API Explorer but not with our application. With our application only feeds we see response are friend request accepts and likes to some other public posts.
3. For another ID Graph API explorer is giving empty results and out application key is giving likes etc as observed in above case.
4. For one of the Use we asked him to turn off the Platform, we were getting messages before. When turned off we stopped getting messages. When turned on we didn't get all messages but only likes etc.
5. When we use Social Mention (socialmention.com) and search for keyword, we could see all messages.

We feel it has to do something with the Application and access key permissions but couldn't get much details. Any help in cracking this is welcome.

Update (May 05, 2014):
The issue is with the way access token generated.

While generating access key we can select permissions to match application requirements. Any thing beyond basic permissions require permission from User. For example if we choose user_location permission, user need to provide permission to get that information. If no additional permissions selected, application will not require any permission and can access all the public data. But if any additional permissions are selected and user not given permission, we cannot get even the public data of that user. With default permissions I generated access key and could get all public data as expected.

"Apache Flume: Distributed Log Collection for Hadoop" - Book review

We have been working on Apache Flume for quite some time now. We used it to load data from Social network into MongoDB and also for log collection. Recently I read a book on Apache Flume titled "Apache Flume: Distributed Log Collection for Hadoop".

 

This is a good starter material for a serious Flume developer. 2nd Chapter provides a good step by step guide to setup and get running with Flume. Liked the way the flow is presented and the mention about important arguments that can be provided to Flume command line. Other useful information from this book is about the Monitoring tools. Though the description is brief good to have introduction to some tools. Overall book gives good details with examples on Flume flow and architecture which includes Channels, agents, sinks, interceptors etc. It would have been more helpful if last two chapters were elaborated more.

Modes of Big Data Analysis

We can look at analysis in three modes based on trigger for analysis.

• Offline/Batch Mode
• Analytics performed and results are made available for applications to use
• Ex: Clinical Trails, Voice of Customer
• Real Time – OnDemand
• Analysis done and results are presented when requested.
• Ex: Up-sell/Cross-sell
• Real Time – Stream based
• Monitor streaming data (Twitter messages, Transaction logs, data from Sensors) and trigger analysis based on event/data.
• Ex: Monitor and analyze online transactions for Fraud, Monitor social media messages for serious incidents.

 



And below are the implementation approaches:

• Massive Parallel Programming (Data Bases and Programming)Hadoop MapReduce
• Scalable Database – NoSQL databases and Databases with ability to store huge data (Ex Oracle ExaData) and to perform operations on data.
• In-memory Analytics - an approach to querying data when it resides in random access memory (RAM), as opposed to querying data that is stored on physical disks.
• Big Data Appliance - combination hardware and software products designed specifically for analytical processing.
• Processing in Memory (PIM) - a chip architecture in which the processor is integrated into a memory chip to reduce latency.
• In-Database Analytics - a technology that allows data processing to be conducted within the database by building analytic logic into the database itself.
• Real-time Stream Processing & CEP

Combination of above approaches need to implement the Analytic Apps

 

Almost 2 years back, for couple of months, I had my first stint with Big data and Hadoop before moving on to Social Analytics. As I resumed my interest into Big data I was looking at my old work and above are from one of my early presentations.

Klout4Java : Java wrapper for Klout partner API

One more contribution for Open source. Uploaded klout4java, a Java wrapper for Klout Partner API v2. I had calls to klout API in our Social Analytic platform. As v1 of klout API getting deprecated, I had to upgrade and I couldn't find any Java wrapper for v2. This lead me to write a wrapper for myself and I uploaded the same for community use.

The link to klout4java http://code.google.com/p/klout4java/

Jab Tak Hai Jaan - Twitter Metrics

One of the topic that attracts good twitter traffic in India is movies. And if the movie is a much awaited Diwali release from Super star, then its a simple expectation to have a good twitter buzz. And on its release day Jab Tak Hai Jaan was a trending topic. I did collected the tweets related and in this blog will present some metrics. Some might call this Analysis but I prefer metrics/report because no Statistical or NLP algorithms involved to derive these results.

Data was from 11/12/2012 18:33 to 11/14/2012 12:15. Twitter Steaming API is used to collect the data and the keywords that were tracked are "JTHJ", "Jab Tak Hai Jaan" and "SRK". Not much of data cleaning applied as these keywords are not so common. Total tweets collected are 104,654 of these Retweets are 31177 (30%).

First lets see simple report, the tweet time line. The graph shows the tweets trending. The value on x-axis is dd/yy HH. Around 3'O Clock in afternoon of 13th Nov is the peak.

Below table shows top sources for traffic with re-tweet segregation. I expected percentage of re-tweets will be more for mobile devices. But my assumption seems to be wrong from below data. Surprisingly, twitter for Blackberry is more popular than iphone though the margin is less. This data also signifies the dominance or abundance of Android.

Source	Total tweets	Original	Original%	Retweets	Retweets%
web	38043	24635	64.76	13408	35.24
mobile web	14813	10382	70.09	4431	29.91
twitter for android	14359	9285	64.66	5074	35.34
twitter for blackberryâ®	8828	6698	75.87	2130	24.13
twitter for iphone	8035	5287	65.80	2748	34.20
facebook	4682	4658	99.49	24	0.51
tweetdeck	1490	1069	71.74	421	28.26
tweet button	1429	1398	97.83	31	2.17
twitter for ipad	1276	807	63.24	469	36.76
ubersocial for blackberry	1114	935	83.93	179	16.07
tweetcaster for android	1103	774	70.17	329	29.83

Most of the tweets with triple figure  re-tweet count are from celebs. Of the top 10 3 are from Sharukh, 3 are from reviewers (like taran adarsh). One from Yash raj films also in top 10. Looks the movie team engaged themselves well in Social Media, which I feel is very important for successful social media marketing. Below are top 5 tweets with max re-tweets.

Tweet	Retweet Count	Author
RT @iamsrk: Very touchng how all of u r buyng tckets for jthj and excted about t. It makes me feel very wanted & postvely pro ...	1221	iamsrk
RT @bipsluvurself: Jab Tak Hai Jaan mkes u wnt 2 find ur soulmate ur true love.Infuses romance in2 ur life.A must see!Saluting the King  ...	641	bipsluvurself
RT @priyankachopra: As everyone comes together to Celebrate Yashji tonight..I sit far away and thank him for touching all our lives..wis ...	567	priyankachopra
RT @iamsrk: En route to Jalandhar for JTHJ...havent slept a wnk all nght...eyelds r feelng lke a mattress. Hope ts a long slow flt...	561	iamsrk
RT @taran_adarsh: Those asking, yes, all three Khans - SRK, Aamir & Salman - were under one roof for #JTHJ premiere. Premiere and po ...	493	taran_adarsh

In the next blog will try to interpret actual review of the film from Tag Cloud generated from these tweets.