RK's Musings

Useful tool

I have often wanted to underline statements / words in a pdf but the security settings of most of the online documents prevented me from doing so. I like the content being in a word doc .

Found a nice software (free) which converts a pdf in to word doc. I am pretty happy with the quality of the output.

Link : HelloPdf

WHY ? ..

NYT article says :

The F.T.C. said that beginning on Dec. 1, bloggers who review products must disclose any connection with advertisers, including, in most cases, the receipt of free products and whether or not they were paid in any way by advertisers, as occurs frequently. The new rules also take aim at celebrities, who will now need to disclose any ties to companies, should they promote products on a talk show or on Twitter. A second major change, which was not aimed specifically at bloggers or social media, was to eliminate the ability of advertisers to gush about results that differ from what is typical — for instance, from a weight loss supplement.

I am definitely against this!. Shouldn't crap reviews/content written , be killed by the "wisdom of crowds" funda. Why is there a need for a regulation ?

Trivia : Why choose PNG over GIF in an app ?

Never bothered to think about this aspect until today, when the choice of the format made a ton of the difference to the output!

Via C# Corner :

First off, why use PNG instead of GIF? Probably the most important reason is that GIF supports a maximum of 256 colors. The second reason is Unisys has a lockdown on the rights to LZW compression which the GIF format can, and usually does use. PNG overcomes the color depth hurdle by providing up to 48Bpp (bits per pixel). Just as importantly, the PNG format is patent-free and available for use by anyone.

PNG has other advantages over GIF as well:

• Alpha Channels: GIF supports binary transparency, that is a pixel can either be opaque or transparent. PNG allows full variable transparency with 254 levels of partial transparency. I should also say that it is also possible to obtain 65,534 levels of transparency, but that is out scope for web graphics.
• Gamma Correction: This allows authors the ability to correct the differences in an image that arise from being viewed by different monitors. Most notably is the difference between PC and Macintosh.
• Two Dimensional Interlacing: Also known as Progressive Scan. While GIF does support this, PNG allows a better representation of the image to displayed as the data is transferred.

Filehash

Handling large amounts of data in R is tricky as R typically loads the entire dataset in to RAM. While this means that computations are going to be very fast, it also means that dataset that can be used for analysis is dependent on your RAM. One solution which I stumbled on was Filehash which seemed to offer me a solution than the usual way that I was going about. I was getting the entire stuff in postgreSQL and then doing computations using R.

Filehash seemed to be great as I don't have to bring another RDBMS in to the work loop. However after spending 1-2 hours on it, I realized it was not that useful for me. It is great if you have key value pairs, but not really great if you have too many columns with dependencies and there is a lot of subsets that you need to work on. In any case, filehash was nothing but DB2 sitting at the back of R. Why should I put my data in to DB2, when a powerful postgreSQL is at my disposal. I have ditched Filehash. There is another package called ff , but sadly it is more useful if you want to sample stuff from a big file. In summary , it appears to me that the only way out is Hadoop-R/Mapreduce-R. That's a steep learning curve for me.

Why should one think of fancy systems like hadoob?
Fact :On a typical day, 6 lakh NIFTY futures are traded in 20,100 seconds of a trading day ,meaning, 30 trades per second.Now each of these 30 trades happen at various price points and quantities. If you want to track the average futures price(quantity weighted) every second for a SINGLE day , it is easy. If you want to do it for a MONTH, it might take some time. If you want to do it for an YEAR and also calculate a 5 minute moving average of futures price, taking in to consideration gapups and gap downs across days, we are talking abt lots of computations on lots of data points. Even in such a simple application using RDBMS systems is a PAIN. I am guilty of using them now, but it will be of no use as things scale.

Future of Search

Via Googleblog:

Marissa Meyer on the future of search

Postfix and Prefix Operators

C++ is a language that requires sustained effort of programming for a years , say, the gurus of the language. May be that is the case..Personally , the language is fascinating in terms of design. Just by looking at why some things are done the way they are in C++, the learning can be immense.

For example - Postfix and Prefix operators -- i++ and ++i . How do you think such a thing is implemented? As you know that a function is overloaded based on the input parameter types. Did you realize that there is no input parameter , at least, on the surface of it ? Then how does it work ? How does compiler know the difference between ++() prefix and postfix..Well, here is something that one can understand and appreciate how things are done. In the case of postfix, compiler silently passes int 0 as an input and in the case of prefix, no parameter is passed.

In our hurry to program away to glory, sometimes, these little inner circles are missed out by us. Do you know that the return type of postfix is a const type and return type of prefix is a reference type..Why in the world was C++ designed like that ?

Well, answer lies in a simple statement , i++++ is illegal , which most of us agree. Double increment doesn't work. WHY ? because i++ returns a const and nothing can be done on const

So, you see, there are some differences in the way postfix and prefix is designed : Return values are different, Compiler passes an input silently in one of the operators and there is a creation of temporary object in postfix.

What's the use of knowing the above stuff ? When you write your own classes, you will keep all these issues in mind when designing your prefix and postfix operators for your custom classes.

Java - typesafe ?

Do you think Java is a type-safe language ? Read Upenn's professor's take on it

Quantitative Finance C++ Library

CODE LIBRARY

Have just started organizing the code that I have written over the last 12 months. Sourceforge is the best thing to use for such purposes. Here is the link to 100 odd classes that I written for pricing derivatives. Need to comment Finite Difference code and upload it . May be I will do it over the labor day weekend.

Link to my QuantFin Library : Source Forge:Quantitative Finance C++ Library

Gruff

If you ever want to analyze data using charts , and , excel -data interaction becomes overwhelming, then one needs to write one's own charting infra..I was in a similar position where excel based charts were not enough for my analysis. There was simply too much data to look at and hence ended up writing my own diagnostic tool using rails-postgres . Gruff a charting library for rails proved to be INVALUABLE.

Link : Gruff . However if you want a little more jazz to the graphs you can look at this link Vertical Labels

Overall, I am happy with gruff for it met my requirement. Don't want anything fancier than what gruff can provide.

Ruby in 30 minutes

I have a problem with Ruby. It is a great language , extremely good for fast prototyping. However this is not the language I use it daily. However, there are times when I have to use ruby where the effort of doing something in a different language is not worth it.

My problem stems from the fact that I forget the syntax as I don't use it regularly. In the last 1 year , I would have probably worked on 4 RoR mini-projects and in every instance I had to pain stakingly go through and review the syntax . This time again I have to develop a diagnostic tool, very very fast and I don't have time to go through the syntax at leisure. I did something different. Picked up this book "Making use of Ruby" and put myself a time limit of 30 minutes for reviewing the relevant syntax..All I can say, is that, the result is pretty good...In 1/2 hr time, one can review the syntax of loops, classes, namespaces, mixins, file operations , instantiations, arrays, hash etc and I must say this book is pretty well organized in that sense. First 120 pages is that would be needed for quick prototyping.

My next task is Rails recap. I just want a quick way to look at price-s score-long/short position data all in one web page and the data is in postgres.The only thing left to figure out is : How to dump the data from postgres in to a chart ? I just hope I figure this out quickly so that I can investigate the results. Sometimes I guess one needs to have patience and develop a diagnostic tool for simple excel based charts are of no help in understanding what's going on...

Quote for the day

"Ease yourself into the language. Don't feel you have to use all of the features, and don't try to use them all on the first day."
- Bjarne Stroustrup on C++

Splendid team effort

Mitchell Baker on his experience at Firefox(record for the highest no of downloads in 24 hrs - 8 Million):

"Every once in a while — for those people who are really lucky — we get to experience a moment where everything comes together. A period where dreams and hard work merge together with remarkable results.

This is such a time for Mozilla.

It’s based on hard work and execution of course. The number of people who have done something unexpected in the last few months, something that changes the outcome, is very high. But that’s only part of it. And there are plenty of times in life — most of life for most people, in fact — where people work hard and pour themselves into their effort but don’t experience the lift and buoyancy of sense of validation.

The periods that are so memorable often involve a team of people, and something that makes that group of people cohesive and satisfying. Sometimes these periods involve working on something that seems giant, hard to achieve and meaningful. Often then involve many things coming together in a way almost didn’t seem possible. And they involve a response from the world at large that demonstrates all the work and energy was worth it."

Blog graph

Analysis of Blog graph by CMU researchers : Interesting observations

Temporal patterns: For the two months of observation,we found that blog posts do not have a bursty behavior;they only have a weekly periodicity. Most surprisingly,the popularity of posts drops with a power law, instead of exponentially, that one may have expected. Surprisingly, the exponent of the power law is -1.5, agreeing very well with Barabasi’s theory of heavy tails in human behavior .

Patterns in the shapes and sizes of cascades and blogs: Almost every metric we measured, followed a power law. The most striking result is that the size distribution of cascades (= number of involved posts), follows a perfect Zipfian distribution, that is, a power law with slope =-2. The other striking discovery was on the shape of cascades. The most popular shapes were the “stars”, that is, a single post with several in-links, but none of the citing posts are themselves cited.

Generating Model: Finally, we design a flu-like epidemiological model. Despite its simplicity, it generates cascades that match several of the above power-law properties of real cascades. This model could be useful for link prediction, link-spam detection, and “what-if” scenarios

Link : www.cs.cmu.edu/~jure/pubs/blogs-sdm07.pdf

Inspiration behind Design Patterns in Software

The Timeless Way of Building

The Timeless Way of Building is a 1979 book that ties life and architecture together, written by Christopher Alexander.

It is considered by many to be one of the major, seminal texts of the twentieth century.[citation needed] It has had a huge influence on creative thinking, especially in the areas of architecture and software design. It could be classified as one of what the Estonian researcher of the Orient Linnart Mäll calls the humanistic base texts.

In the book, Alexander introduces the concept of the "quality without a name", and argues that we should seek to include this nameless quality in our buildings. This quality is hard to pin down in words. Alexander uses the technique of surrounding it with existing concepts that reflect a part of the quality with no name but are not sufficient to define it individually.

One central concept in The Timeless Way of Building is that of creative unfolding. Alexander uses the extended metaphor of the growth of a human embryo - with illustrations - to take the reader through a creative complexification process.
This starts with a single orientation (in building, say, where the new building will be and which way it will face; in the embryo metaphor a single fertilised cell) and gradually unfolds (in building through the thoughtful addition of new features - where the entrance will be, what sort of root it will have; in the embryo metaphor how the spine forms and where the arms and legs will be) into a fully featured, complex yet somehow naturally appropriate design. A completed building; a new born child.

This book popularised the concept of the pattern language.

Link : Book

Bridge Pattern

Via Bridging the gap between Interface and Implementation :

A nice article explaining the importance of Bridge Pattern

Useful Tools

Via : 15 -hot- tools that made me a coding Paris Hilton.

Tivo for India

Why shouldnt there be a Tivo for India ? Why hasn't been there one ?

• Is it because of purchasing power of consumers is limited and would not be willing to invest in an instrument which records their programs of interest ?
• Out of overall 200 Million TV set penetration in India - Rural folks dont mind TV ads
• Marketers still rely on TV for promotional activities and customers are often hooked on to ads. I have seen a lot of folks discuss some of the quirky, catchy ads being discussed over the  lunch table. Does this mean TV as a mode of ad promotion is going to stay for a lot more years
• What will happen to the mobile devices ? Additional connections are 6 million per month . How can one leverage this platform for promotion
• Recent development in terms of sms promotion from theatre services , airline ticketing services, insurance, banking services show various sectors are trying to grapple with the challenges of promoting their product in the 160 characters of an sms

So, can one rule out Tivo type of product in India ?

Mobile Search $20B Mkt in the coming 15-20 years

Mike McCue , cofounder of TellMe networks on Voice Recognition and Mobile search..

Via  Fast Company:

FC: What's the potential market for voice-powered applications?
McCue: In the enterprise segment, it's a multibillion-dollar market. Directory assistance is another billion or two. That could grow by $10 billion or $15 billion in 5 to 10 years. Personally, I love the iPod (NASDAQ:AAPL); I have about 30,000 songs on mine. But it's impossible to navigate with that wheel. It's crying out for speech interface. That could be huge.

FC: What about mobile search?
McCue: That market is now basically zero. Over the next 5 to 15 years, it will be a $15 billion to $20 billion market.

FC: How will that be commercialized?
McCue: It will be ad supported. It's going to take time to develop, but it will happen. We will handle the design, but Microsoft is doing a lot of work with ad engines, ad inventory, and the relationships with advertisers

Emerging Culture of short attention spans

Here’s another evidence of how our attention spans have reduced and firms are catering to that demand.

Sony Television is retooling old series like “Charlie’s Angels” into episodes of under six minutes.

Flickr Id

Flickr Id has immense use in web apps. If one needs to have a gallery that has to be displayed on the website, embedding Flickr slide using Flickr Id is the easiest way . Use idGettr - Find your Flickr ID  to  find your Flickr user or group id.  I am developing a website for an NGO in Bangalore. Displaying their past events using flickr was immensely useful to me. No longer one needs to copy all the images in public/images folder and create gallery layouts in your webpages..Thanks Flickr!

Here's the code for embedding a slide show
<iframe align="center" src="http://www.flickr.com/slideShow/index.gne?
user_id=12345678@N00&tags=YOUR_TAGS" frameBorder="0" "width=500" height="500" scrolling="no"></iframe>

Funding numbers

Internet and Mobile Startup fundings in India
2005 - VC investment in India - 7 deals - $35M
2006 - VC investment in India - 30 deals - $200 M
2007 - ??
any guesses ? Travel, Matrimony and Jobs are the regular ones that are making the splash..Whats the big thing that will kick off in India ? food for thought
 

Barcamp 3 Bangalore

I was at Barcamp3 Bangalore today , the second day of the barcamp.Some of the sessions that I managed to attend were

Tinkering Culture OR How Intellectual Property is Screwing Us : A superb and riveting talk about the IP by Mr. Lawrence. The breadth of examples presented in the session was mindblowing. The talk was focused on umpteen number of cases where read web was becoming read/write web. The session concluded with the thought Rescension (Constant Retelling of old stories) was the way to go where every individual should have the right to recast the product / service and serve up to customers.

Pickle Life Moves Pickle It. Photo and Video Ramesh Babu: A demo of photo and video sharing site - http://pickle.com/

Yulop - My take was that it wont fly unless they focus on one thing. The concept was called citycasting where in the entire city would be on the net. The content would be generated by the users and the services included a host of services , and here is where the problem lies. I guess the service would make sense only if the founders focus on only one service.

The Knowledge Foundation: Opportunities, Ideas, Challenges : This was a pitch by a group of professionals ,started a not for profit firm which conducts workshops, seminars, conventions. Barcamp is also their local adaptation of the US Barcamp

Can the new economy people cause a cultural shakedown in India ? - Subbu Vincent (India Together) - This was a very interactive session where the discussion revolved the challenges in starting a business and the way the new economy folks can create a cultural shakedown in India. The conclusion of the session was that we, Indians equate disagreement to disrespect and try to acquiesce with a lot of things which have to be challenged.

How to make users love your application / website. : A pathetic session which had no structure and content worth listening to

Big Fight - AJAX vs. Flex  : It was not a big fight as the title suggests.It was more of a casual discussion on AJAX and Flex. Flex was completely new to me and it was a nice learning session.Flex is a developer friendly platform

BAMBI: case study and Discussion on innovation in big companies  : This was a nice case study that showed how codehack was adopted in GE Medical systems and getting the employees unleash their creativity.

Social Networks in India - What works and what doesn't : The session was about 3 aspects
    What makes indians use orkut and not not minglebox/hifi. ?
     What  are some of the success stories in India ?
    What kind of things can one looked forward ?

Overall, the gyan sessions were superb, demos were okish. Overall, a nice experience. I look forward to the next barcamp

Online Display Ad Market Space

An thought provoking article on Online Adspace : The quest for the perfect online ad 

IBM - India focus

IBM India employee strength is mind numbing..Out of the IBM - Total base at the end of 2006 - 355,766 people, India has 52,000 people ( up from 36,000 one year earlier, three years earlier, IBM had just 9,000 people )

15% of one of the biggest companies in the world is in India...Wow!! with this kind of talent, when will there be a superb product from India ?

Video search - advertising

Lot of firms are trying to crack the video advertising puzzle ..to become the next google in video advertising..Firms that have making soe noise are

• ScanScout
• PodZinger
• Corp
• Blinkx
• Brightcove Inc.

Lots more to come I guess..It will be interesting to see which firm would finally crack the video ad puzzle

Indian ecommerce market - Ad earnings

From eStatsIndia.com,Indian e-commerce market - b2c and b2b combined - is about Rs 4,100 crore( 1 billion usd). Let's compare with the global ecommerce market.Online shopping accounted for 3% of total retail sales in 2006, nearly double the share from 2002. E-commerce sales totaled $108 billion in 2006 with an expected 20% increase for 2007. E-commerce sales are forecasted to climb at a double digit pace for the remainder of the decade, accounting for 4.3% of total sales by 2010. (Source: US Census Bureau and Business Week)  . In comparison , there is a lot of catching up that needs to be done by India...Ok now back to India numbers

B2C
Numbers in crores :
Retail: 150 (billing figures)
Jobs: 135
E-greetings: 12
ISP’s (domain booking and other related services excluding hosting): 10
Matrimonial/Dating: 40
Travel (Ticketing): 750 (billing figures)
Advertising: 162 (revenues)
Broking: 500 (these figures are commission earned by the online arms of the broking firms)

Converted to % fig , travel amounts to 43% of industry and broking 28% ...Advertising - 9%.

What's the scope for increasing the pie for adv. revenues --- on which Sites like guruji depend on ?

Cisco and the acquisition game

Cisco starts its acquisition game , now in social networking space. Its recent acquisition is
Five Across 

About the firm
Five Across is a leading provider of social networking and community building technology that helps organizations ranging from small businesses to large enterprises connect to their community constituents and customer base. Connect, the company's flagship product, is a rapid-deploy web platform enabling organizations to strengthen and grow user communities with dynamic multimedia experiences. Organizations using Connect can grow user demographics within their brand and build a profitable business model with on-demand and user-generated content.

Steve Jobs Call

Will the Music Industry go with Steve Jobs call i.e No DRM ?

Adsense triumph

The news item was indeed the validation of long tail in advertising..62% of the revenues came from adsense as compared to 25% couple of years back. Reminds me of Goto.com case where the revenues from syndicate business reached 90% of their total and the management for some reason decided to concentrate on syndicate business ..Thus goto.com as a search destination slowly vanished...Such situation will never happen with google..or will it ? What if 99 % of the revenues come from adsense ? What will be the fate of adwords ?

Google's net income nearly tripled to $1.03 billion as the Internet giant continues to gain market share in Web search and online advertising. Revenue rose 67% to $3.21 billion, including $976 million in commissions paid to marketing partners.

Revenue growth on Google-owned sites outpaced revenue from its Adsense program. Google-owned sites generated revenue of $1.98 billion, or 62% of the quarter's business.

Flexible thinking

Sometimes people wonder why there is Struct and Class in C++ language. They have identical functions except that all the variables defined in struct are public by default and those defined in C++ are private by default. The need for both of these to coexist seems redundant. This is where one needs to go the history of c++. The struct was very much a part of C language and to evolve in to a much richer form, Class was an invention so that it would give a choice / flexibility for people who wanted to stick to struct or change to Class. This thinking was on the assumption that the evolution of class would not be hindered by the syntactical restraints of Struct. This thought was that of Bjarne Stroustrup, the inventor of C++ in 1979 at Bell Labs. History has proved him right!

Simple flexible thinking led to the evolution of the entire evolution of classes and object oriented programming. In our thinking too I believe, if we try to accommodate a little flexibility in our thinking about the tools we develop, the analysis we do, it will lead to results beyond our imagination