hacks

The Cost of Cowardice

Alternate title: How I Learned to Stop Worrying and Love the Bomb

I am a fairly risk averse person, which often leads me to making sub-optimal decisions and living in a little bit of regret. I also try to be as rational as possible, which leads me to analyze every little or big decision that I want to make, and in most cases eventually arrive at a good understanding of my choices. This then devolves into a self harassment because I am unable to take the riskier routes, even if they overshoot in the Expected Value department.

This can have pretty serious long term consequences, depending on the decision type. Let’s take two seemingly unconnected examples to drive home the point.

A. Relationships
Relationships are tough. They are difficult systems to analyze because you are both the observer and the observed, and your observations will affect the system you are trying to rationally figure out. So when trying to do fault analysis on your own relationship, you have to be aware of the fact that any reflexive process will only lead you to an incomplete truth. That does not mean that honest analysis of such a situation is useless, it only means that the inherent uncertainty associated with whatever conclusions you make can throw off your confidence and lead to a lot of second guessing (in a lot of cases, it should).
Let’s say you analyze the issues in your relationship and arrive at some reasonable conclusions that seem to be holding up in both back and forward time testing. In a lot of cases, the conclusions you arrive at will have you facing some tough choices, like ending a relationship, or changing important things about it, and/or having some difficult conversations.
In such cases, what do you do? The optimal choice is in front of you, but there is hesitation in your mind because you are not totally certain (and you’ll never be) of the quality of your decision. And since it is a difficult conversation to have, you chicken out. The cost of prolonging such decisions are two fold:
a. you stay miserable and the issues never get solved, and usually the pain only intensifies with time (compounded pain, we can call it)
b. in case you have arrived at an incorrect conclusion, there is very little correction that you can do unless you get more information. The best way to get more information in a dynamic two (usually) people system is to confront the other person and get them involved in the process as well

B. Finance
The other case is money. Money is a tricky beast, you never know if you have too little, too much, just right, but the one thing that’s certain is that you would always like more (assume no change in effort). Money is also the place where you get one of the most objective feedback of your decisions- you will have very little if you are not making or saving enough. You can also veer off in the other direction where you are stashing way too much and not putting it to proper use and still living like a poor person with no way of ever attaining full financial freedom (where you don’t need to work for money because your money makes enough money for you).
In my case, it took me way too long to get the courage to move my money from a shitty 0% yield savings account to a mix between investing and deposit in a high yield savings.
The costs are clear, because with investing, the most important thing is time. The more time you give your money to grow, the more it will grow (compounded gain, as many call it). Again, the fear of the unknown, even though it is well documented to be the better choice, leads you to sub-optimal decisions that cost you real money in the long term. It can be massive amounts, a couple year’s wait can cost you as much as 50% of what you didn’t invest in that time (over 30 years).

(Shameless advertising alert:
By the way, Wealthfront is where my money is at, and they make it easy to get started in case you are also stuck in a similar position as the old me. Sign up using that link and get yourself $15K managed for free)

Where do we go now but nowhere?
Once I realized that I had been falling into this trap over and over again, I had to figure out a way to escape it. My solution isn’t ingenious in any way, but it doesn’t have to be. The thing is that I only have to perform better than the significantly worse things that I just described above. It would be great to perform much better though, and I have devised a plan for that as well.

The solution is borrowed from most desensitization practices. The idea is to slowly expose yourself to what you are afraid of in minute quantities at first, and then slowly increase the exposure. So if you are afraid to talk about a touchy personality flaw of the person you are in a relationship with, try slowly introducing them to the idea by occasionally pointing it out in the context of other relationships or your own. If they are receptive, it can be over with before you even realize. In case of money, start by saving a fixed small amount over your current saving every month if you can. If your problem is lack of investing, try investing small amounts at the beginning with tools like Acorns, Wealthfront or Betterment. Over time, as you get comfortable, increase both the frequency and the amounts.

Where you can really accelerate into hyperdrive is to draw parallels between these seemingly different situations and accelerate the rate of your desensitization by not only having a regular speed of increased exposure, but a regular speed at which you increase the speed too. You will realize that you will gain confidence to take more risky bets (with higher Expected Value) in general and that will result in you wasting less time in other spheres. If this connect isn’t drawn, you will be “reinventing the wheel” (definitely the most cringe worthy cliche phrase uttered on this blog) every time.
When you realize that there isn’t much to lose when you invest in the markets for example, and the only thing that is preventing you from taking that step is a fear/dread/aversion of the unknown, you realize that the same thing is preventing you from asking out that cute barista who always seems to be super nice to you, and you can only come to the conclusion that life can only get better if risks with low downside and high upside are embraced.

Go. Do.

Citibike: Analyze yourself!

In the 5 years that I have lived in NYC, the biggest improvement anything’s made to my quality of life has been Citibike. I like being on wheels, I like being energy efficient, I like the wind in my face, I like my freedom of commute (a regular bike binds itself to you), and I like that it saves my trip data! I like to think I am a Fastridahhh, but we are not always what we think we are. So by baselining myself against this analysis, I wanted to see where I actually truly lie.

The task was simple, get the lat/long of my start and end citibike stations, use Google Distance Matrix API to figure out the biking distance between the two locations, use the time from my citibike data, and get my average speed and see how it changes with distance etc.

Once the stew was cooked, the results were fairly interesting.
My average speed was 13.7 kmph (8.5 mph), but I have done no outlier removal or anything because lazy. The median was pretty much exactly the same as average (13.56 kmph), indicating an even distro. This was a little disappointing frankly, I was only slightly faster than the overall average according to the wider analysis (8.3 mph) and my self image rightly got punched back into its place. I can make up excuses for why it is slower than I feel I ride at, but that is just being dishonest (I have already thought of three, including shifting the blame on someone else).

Most of my trips are short distances that are annoying to walk (1-2 km), with nearly 50% falling in that range, the rest are equally distributed across the other buckets.

Distance Range # Trips Average time (min) Average Speed
0 to 0.5K 8% 1.53 8.24
0.5K to 1K 21% 3.87 11.90
1K to 2K 48% 5.89 13.09
2K to 5K 22% 11.31 14.15
Greater than 5K 1% 16.12 18.84
Grand Total 100% 6.38 13.36

I also wanted to look at how speed varies with distance, as you would expect it to be highly correlated (they were, with a correlation coefficient of 0.4). But I also created a polynomial best fit line with an order of two (anything higher is just overfitting and the data seems to follow a parabolic pattern rather than linear). The R-squared sits nicely (for such unclean data with so few datapoints, especially for higher distances) at 0.265, which vindicates my hunch about it being a parabolic curve. Here’s the pretty chart for your pretty eyes to peruse:

Speed v Distance

If you want to do it too:
Please do, and let me know what your results look like.

Getting the lat/longs:
Surprisingly, Citibike doesn’t have any place that you can get the lat/longs of stations from unless you distill it from their trip data. So I did that. And here’s the file for all you guys (updated till March 2017, includes Jersey City citibike locations)
Citibike Stations Lat Long

Using the Google Distance Matrix API:
I prefer Excel for small analysis, so wanted to write a VBA code that would get me the distance between two lat/long pairs. Here’s what I made, with a little help from my friends over at StackOverflow given that I don’t really know VBA. You can create a new module, paste this code there save it, and then use the function called getDist to get the distance between the lat/long of the locations (which you’ll need to vlookup from the station lat/long map that I shared earlier)

(WordPress is being a prick and not letting me format this properly, so please insert tabs at appropriate places)


Public Function getDist(orig_lat As Double, orig_Double As Double, end_lat As Double, end_long As Double) As String

Dim xhrRequest As XMLHTTP60
Dim domDoc As DOMDocument60
Dim domDoc2 As DOMDocument60
Dim nodes As IXMLDOMNodeList
Dim node As IXMLDOMNode

'You must acquire a google api key and enter it here
Dim googleKey As String
googleKey = "XXX" 'your api key here

'Send a "GET" request for place/textsearch
Set xhrRequest = New XMLHTTP60

Dim requrl As String
requrl = "https://maps.googleapis.com/maps/api/distancematrix/xml?origins=" & orig_lat & "," & orig_Double & _
"&destinations=" & end_lat & "," & end_long & "&mode=bicycling" & "&key=" & googleKey

xhrRequest.Open "GET", requrl
xhrRequest.send

'Save the response into a document
Set domDoc = New DOMDocument60
domDoc.LoadXML xhrRequest.responseText

'Find the first node that is called "distance" and is the child of the "result" node. The SelectSingleNode will pick the first result, as google can return multiple options
Set ixnlDistanceNodes = domDoc.SelectSingleNode("//distance/value")

If Not ixnlDistanceNodes Is Nothing Then
getDist = ixnlDistanceNodes.Text
Else
getDist = -1
End If

Set domDoc = Nothing
Set xhrRequest = Nothing

End Function

EPL vs La Liga: Which league is better?

After numerous subjective discussions (heated arguments) with friends of mine comparing EPL to La Liga that could only serve to break friendships and bruise throats, I decided to actually do some work and analyze the quality of the two leagues.
Disclaimer: I really like La Liga, and I really like Barcelona.

There are two main equally important components of the quality of a soccer league:
1. Strength of the top teams: There may be soccer leagues in India and Egypt with fierce internal rivalries and healthy balance in strength, but if the top teams itself are weak, the whole league is weak. To paraphrase Radiohead, the best they can ain’t good enough.
2. Competitiveness: This is the reason there is no Bundesliga in this discussion. A league is fun only if the day to day is exciting.

Intuitively, point 1 goes to La Liga, while point 2 goes to the EPL. But let’s allow stats to join the discussion.

I collected the final season tables for the two leagues from 1997-98 to 2013-14 in order to study the aggregate over these 17 seasons. Now some people may say that this is not recent history, but if you really want to talk stats, you need to have more data points. Otherwise the point of this article is moot and we can all go back to arguing relentlessly.

Also, I defined something called Nu Points, because I don’t like the current point allotment for WDL (3:1:0). I either want to make it more balanced (4:2:0) with every match getting 4 points (Nu Points), or more win heavy (4:1:0) with every win adding more weight in case of an imbalance (Wh Points).

So here is the aggregate table, and it is a counter intuitive curveball:

Counter-intuitive poop

Aggregate Table

Standard deviation is a metric of competitiveness, the lower it is the tighter the league and fiercer the competition. And by the cock of Zeus, it is a result we did not expect to see! La Liga has a lower standard deviation compared to EPL. But we all know that EPL would be more competitive, so what the hell is going on?

The answer lies in the details, and how aggregated statistics can sometimes obfuscate the story than reveal it.

Here is the breakdown of the team wise statistics for the Nu Points and Wh Points for the top 20 teams in each league (top teams defined by the number of seasons played, followed by average points).

EPL Nu Points

EPL Nu Points

La Liga Nu Points

La Liga Nu Points

EPL Win Heavy Points

EPL Win Heavy Points

La Liga Win Heavy Points

La Liga Win Heavy Points

And now we see what is going on. FC Barcelona and Real Madrid are dominating the Liga at the expense of the other teams (this is even more stark when you consider the Wh points), but the competition among the rest is very fierce and most of them are even. This contrasts with the EPL, where the top 4 are much stronger (in the vein of RMA and FCB) and there is an unmistakable steady decline in the quality of the other teams. This leads to the higher standard deviation in EPL compared to Liga.

So clearly, with similar coefficients of variance, statistically speaking, both leagues are imbalanced. And no sane person should disagree here. This is a general problem with football leagues as the distribution of funds happens to be in a positive feedback loop which can only be broken by the infusion of big money (Chelsea / Man City) into some of the lower teams. What gives EPL the perception of being more competitive is the closeness of the top four compared to Liga. All of the top four in EPL (now five with Man City joining in with a gorgeous Upper Limit) are quite close and are offered decent shots at the pole position. This is completely unlike Liga where RMA and FCB are thoroughly dominating the minions. Since competition for the top is what matters, EPL is clearly more competitive.

From 1997 to 2013, I would say that the best of EPL has been as good as the best of Liga on average (key here being on average, maybe Liga narrowly edges EPL). If that is an assumption we can agree upon, then clearly EPL offers more in terms of competitiveness than Liga.

Point 1: Tied (Liga edge)
Point 2: EPL wins where it matters

Overall winner, EPL.

Congrats EPL! But trust me, if you want to learn how to play good and enjoyable football, watch the current majesty of Barcelona. I just had to fucking say that.

Scrabble Words Finder

This is a simple VBA macro that I made that can find words with the letters (upto 4) that you want. I was frustrated with the lack of such a software/site that could tell me the possible words that I could make with certain letters. This macro finds such words, and arranges them in the decreasing order of the points they’ll give you in a scrabble or scrabble based game.

The code is included and you are free to make any changes and redistribute it wherever you want. I have hardcoded most things as I am new to VBA, so don’t change the columns and stuff or rename the sheets, unless you know what you are doing. Have fun!

I am not too sure if wordpress allows .xlsm files, so I changed the extension to .xlsx and uploaded this. Please change it back to .xlsm once you download it.
Scrabble Word Finder

This is the .xlsm version.
Scrabble Solver (xlsm)

Features of Pandora One without paying for them (on your computer)

There are two significant disadvantages/annoyances of Pandora if you use it on a laptop in a window:
1. Ads
2. Are you still listening?

To hack the first, just use AdBlock plus (a Chrome, Firefox extension), and uncheck “Allow some intrusive advertising”. This should take care of the annoying ads.

For the second, the workaround is slightly complex. You will have to install Leechblock if you use Firefox or Nanny if you use Chrome. The trick works because when Pandora goes “Are you still listening?” the URL changes to http://www.pandora.com/inactive
In the settings, do these things:
1. Create a rule to block http://www.pandora.com/inactive
Set the rule to run for 0000-2359, and do step 2.
2. Search for the “redirect to” URL and put in the station you listen to on Pandora (or just https://pandora.com since it would recognize your login and work properly). You should be good to go.

Two irritating features removed that would make Pandora pure joy, as it should be. Needless to say, these work only on a computer (no AdBlock plus on Android yet! Boo!). But if you are using Pandora even on your phone, just pay up. The charge is hardly much.