RK's Musings

Quote for the day

“Good or bad, moral or immoral, people are going to make markets and trade via computers, and this is a natural area of financial engineers,”

-Emanuel Derman

Never Grow up :)

Quote for the day

“In three words I can sum up everything I’ve learned about life: it goes on.”

– Robert Frost

Quote for the day

What lies behind you and what lies in front of you, pales in comparison to what lies inside of you.

- Ralph Emerson.

Counter

Quote for the day

 

Mathematics is discovered in much the same way as any other science - by experimentation (here, simulation) followed by confirmation (proof). All too often, students think mathematics was created by divine inspiration since, by the time they see it in class, all the "dirty work" has been "cleaned up."
- Prof Alan Levine

I guess the "dirty work" is where all the fun lies !

Quote for the day

 

You can't treat cancer with pain killers. That's what is happening to US Banking.

- Taleb

Quote for the day

Any poet, even the most allergic to mathematics, has to count up to twelve in order to compose an alexandrine.

- Raymond Queneau

Quote for the day

Noah’s Ark was built by amateurs. It took professionals to build the Titanic.

Quote for the day

Your net worth to the world is usually determined by what remains after your bad habits are subtracted from your good ones.

- Benjamin Franklin

Quote for the day

"A good traveler has no fixed plans, and is not intent on arriving."

- Lao Tzu

On Flow

Via Mihaly Csikszentmihalyi((pronounced "CHICK-sent-me-high-ee") , Chicago professor and the author of one the most influential books,"FLOW" says , being in flow means :

• Completely involved in what we are doing - focused, concentrated
• A sense of ecstasy - of being outside everyday reality
• Greater inner clarity - knowing what needs to be done, and how well we are doing
• Knowing that the activity is doable - that our skills are adequate to the task
• Sense of serenity - no worries about oneself, and a feeling of growing beyond the boundaries of the ego
• Timelessness - thoroughly focused on the present, hours seem to pass by in minutes
• Intrinsic motivation - whatever produces flow becomes its own reward
  

Quote for the day

A programmer can write tens of thousands of lines of code, and produce a lot of software that works. A less productive coder can write a tenth of the lines, perhaps even editing down what she writes so that there’s less code, but they’re better written. This small program might be the most useful thing on many people’s computers, flawless code that just works.
- Leo Babauta

Quote for the day

Quote for the day

All models are wrong, some models are useful.

- George E. P. Box

Quote for the day

Quote for the day

Short term memory

It has been 2 months since I have coded a single line of C# . Today I have to use it again for some reason and I seem to have forgotten everything about it. My short term memory is killing me !.. Every once in a while I face this problem. A month ago, I forgot ruby syntax. It took me almost a day to get back to speed. A few months ago, same thing happened with PostgreSQL syntax. 3 months back with VBA Syntax!!

Sometimes I think I have to live with short term memory thing . Like now, I have to spend god knows, how many hours to get back to speed on C#. I don't know how good programmers maintain their syntax knowledge for something they have not coded for a couple for months. May be they have good memory or they have some good memory aids. Whatever that might be, I seem to be lacking..

Short term memory has also been a boon to me. I don't remember much of the past which kind of makes me feel lighter and approach any subject/ concept with no prior hypotheses...But in situations where you have to code which means you got to remember syntax, I become painfully aware of my handicap! Don't know when I will find out a solution to remember syntax for at least a few languages that I am certain to use in my working life --- Ruby, Rails, C#, VBA, C++, C, R, PostgreSQL .

Quote for the day

We are drowning in information and starving for knowledge.

–Rutherford D. Roger

Quote for the day

Via Jayanth:

The relationship between two members of a family should be like fish and water,

but its should not be like fish and fisherman.

Disconnect Daily

Via ZenHabits:

Have a disconnect time each day. It’s like setting office hours if you’re a professor — you set the times that work best for you, and you can even let people know about these times. Let’s say you are disconnected from 8-10 a.m. each day, or 4-5 p.m., or even anytime after 2 p.m. Tell people your policy, so they know you won’t be available for email or IM. And use this time to create.

Quote for the day

"Human beings don't like to trade against human nature, which wants to buy stocks
after they go up , not down"

- Nunzio Tartaglia (Pioneer of pairs trading)

Even though this technique is Stat arb 101, the advantage of this method is designed more to exploit the investor's over-reaction to the events. 

1000 Posts

I started blogging a few years back, a big thanks to my friend TP who motivated me.... Today Typepad had a message that I have blogged my 1000th post. Did blogging help me? Its a resounding yes because I have understood things better once I had written them down, most relevant in this case are book summaries.

Workout

Nice points about "working out" via Zen Habits

Exercise:

• Do it 4 times per week for at least 30 minutes. Seriously, doesn’t matter. Basketball, jogging, weights, push ups, etc. Unless you have a specific goal just make sure to get it done!
• The two day rule! You can’t miss 2 days of working out in a row. You can skip one day. But not the next!
• Don’t let the “Exercise Monster” build up. When I don’t do laundry for a while it becomes what I like to call the ‘Laundry Monster’. It builds up in my mind and I avoid doing it at all costs. Until finally, I have to. And it’s never as bad as I think it’ll be! Don’t let it happen to you. Follow the two day rule!
• Don’t focus on exercising at a certain time. Just focus on making time to exercise!
• Make it fun! Listen to music; join a nice gym; etc. Bonus: If you love working out with music as much as I do, only listen to your favorite music while you workout. I’ve actually exercised just to listen to my music.
• Focus on how you want to feel. We always feel energized after we workout. Yet, we usually skip exercise when we’re feeling tired! Remember: Exercise will give you energy. Want more energy? Exercise more!
  
  
  
  

Beautiful Poem

To Be of Use
- Marge Piercy

The people I love the best
jump into work head first
without dallying in the shallows
and swim off with sure strokes almost out of sight.
They seem to become natives of that element,
the black sleek heads of seals
bouncing like half-submerged balls

I love people
who harness themselves, an ox to a heavy cart,
who pull like water buffalo, with massive patience,
who strain in the mud and the muck to move things forward,
who do what has to be done, again and again

I want to be with people who submerge
in the task, who go into the fields to harvest
and work in a row and pass the bags along,
who are not parlor generals and field deserters
but move in a common rhythm
when the food must come in or the fire be put out.

The work of the world is common as mud.
Botched, it smears the hands, crumbles to dust.
But the thing worth doing well done
has a shape that satisfies, clean and evident.
Greek amphoras for wine or oil,
Hopi vases that held corn, are put in museums
but you know they were made to be used.
The pitcher cries for water to carry
and a person for work that is real.

Quote for the day

“First we make our habits, then our habits make us.”

- Charles C. Noble

Quote for the day

Parsimony is desirable but not always obtainable

- Edward Jackson (PCA Practitioner)

I guess it's true sometimes, about of our lives too !.

Quote for the day

In God we trust; all others bring data.

—Dr. W. Edwards Deming

Turning a Boring task in to Fun

As we all know, every kind of work involves a definite amount of tedious, unpleasant,low-mid IQ work.I don't subscribe to these labels though.

In the life of a data analyst, the data preparation and data cleansing is something which has to be done before he/she can even think of doing any sane analysis. So, how does one approach data prep and data cleansing? Is it something to be looked down upon ? Well, I was under that impression many years back. My view changed completely when I took the effort to understand the complexity of data prep and data cleansing phases. Also, I began to develop this feeling that working on these phases gives a tremendous amount of insight that no modeling on the cleansed data would give. Here is a fact which is interesting : Renaissance hedge fund employs around 25 top notch PhDs to work on data treatment such as missing and capping treatment, outlier effects etc. No wonder this amount of care at the initial stages translates to tons of insights in to the arb opportunities in the market. Many years working on databases, I have kind of developed great love towards it. One of the things which is a pre-requisite to successful modeling is data prep phase. Working on data, which to this day, many people have an aversion is a fascinating place to start the analysis. However I am amateur in terms of the possibilities that lie in DBMS world. I have to understand and implement lots of things in this space, at least in relation to fin world. KDB+, Hadoob(TP keeps reminding of this fascinating world that I have not explored at all). Well, I guess it is a life long process.

Recently I was asked to quickly develop a database of brokers and sub brokers associated with NSE, as a side-activity when I am not doing my regular work. This is just one of those requests you get when people know that you program . Well, at the first sight of the task, it does seem as though it is a boring task. Well, all it takes is to screen scrape NSE site , dump the content in to delimited text file, read it back in to whatever dbase you want to . Well, if the task is to hand over a xls, just import the delimited text file and be done with it. This task of writing a program to screen scrape NSE was easy. However preparing the data from badly formatted html files is a little painful. As mundane as this task sounds to be, it was a pleasant experience. HOW?

• Haven't screen scraped NSE before,But took this little task as an opportunity to do just that..As they say, all it takes to recognize a good programmer from a bad one is to look at 100 lines of code . Some companies conduct rounds of interview, Phone interview, this type of interview, that type of interview, HR round (what do you want to do in life type of questions, which the interviewer himself would have no clue what he himself would want to do in life, 5 years down the lane questions , how do you see yourself ,,kind of questions for a PROGRAMMING job!! give me break!. In 5 years the skill sets I have could become completely obsolete and may be I will teach math:).. Any ways the point is, IT companies should just ask the interview candidate to code for 15 minutes and examine the code.It gives far better insights in to the candidate's productivity than asking "what are your 3 strengths and 3 weaknesses". Till this date I have never understood how these questions make sense in any interview( For heaven's sake, I am not at a shrink's place where quizzing abt my psyche is imp!)
  
  NSE definitely needs to improve its IT coding standards for the site is not being programmed properly. This is not some casual statement made by hear say. Screen scraping the relevant files shocked me to the kind of coding standards that were being followed!. Investigation of this sort is always fun.

• Had used C++ earlier to screen scrape but it failed becoz the library that I was using was built for HTTP and not HTTPS. Well, I could rebuild it ..but gave it a thought and ditched it to favor C# library. It worked like a charm. C# is a language for people who just want to get some things done with no particular great concern about efficiency, performance etc. At least thats what I believe.There are tons of libraries that are build and you just have to use them. How much time does it take to get up to speed on C# screen scraping?.Not more than a couple of hours.

• Thanks to the poor formatting of the files, had to resort to manual search of relevant tokens. Made it fun by using regex which I had extensively used probably a year ago. But you know, programming is like going to gym. You go to gym regularly and you are fit. You stop it for a month, you become lazy and you stop it for an year,forget about running for even 15 min on a treadmill.Your body will not cooperate. Programming is exactly like that. If you want to just type in with out looking too much in to the reference section of the language, with out Ctrl C+ Ctrl V(which most of the people indulge in, they call it programming, i call it effective copying), you got to program daily. If you don't enjoy it, get out of programming and do something else. So Regex became something which I had stopped using a year ago. I mean I had conveniently avoided it , becoz..well, lets face it...doing it with the dumb Contains() or searching manually appears easier that regex. Using Regex needs you to think..Searching plain text for tokens appeared correct becoz I did not want to invest time in creating regex in the first place. A mundane activity like parsing a ill formatted html file gave me an opportunity to go back to gym( use regex which i had not used since a year) . Needless to say, it was like going on a treadmill after an year. However after spending 3-4 hours on regex, I was back to normalcy :).

• So, here I was with a broker and sub broker database and was ready to hand it over and be done with it. I had spent on this task already close to 20 hours. Then I just looked at the data and thought , what would I want to think about this data and answer some subway ride questions, meaning, if you look at data on a subway ride, and you have nothing else to do but to wait till the train takes you to the destination, what questions would you ask about the data ? This is type of mindset where you think furiously whatever random/rational/senseless questions that you have about the data and quicky answer them. Well, I had so many other things to do ..Should I spend any more time on this broker/sub broker dbase..Then I thought, why not ? This is probably the only thing that I will recollect from this brief task , apart from just the usual hacking recollections. So, I spent an additional hour looking at Data. Just an additional hour greatly helped me answer some questions which will help some back of the envelope calculations about ALL BROKERS that are associated with NSE. Now that's some useful info

• Should I develop a rails interface ? That was a stretch..Have other important things to work on and hence decided to leave this task at this stage and handover the dbase.
  
  Sometimes, I do not know whether thinking in detail about any task hinders productivity or enhances productivity. Had I have not thought in detail about this task, may be I would have done it in lesser time and probably would have not had fun in the process.But one thing I do believe is that thinking about details at a micro^micro level makes any boring job a fun activity.
  
  
  

Quote for the day

"What can be done with fewer is done in vain with more"

- William of Ockham