Use flickr tags -> KML, then apply surface to that.

Found this on a blog post (lifehacker?)

http://mapalist.com/

This is just a note to let everybody know that things are going to slow down for the holidays while I get some post-doc applications together, do some contract work for a genomics/genetics lab and recover from my bike trip through Nicaragua…

This is an open letter to the owner of the linksys WRT54G router on IP 200.91.113.172, specifically who had the dynamic IP on the amnet subnet IP 10.105.97.121 (the sub-network) on Sun/Mon 24 Nov 2008.

This is to let you know that I have made the following changes to your router from the current defaults.  I set the time zone correctly. I noticed that it was not connecting to the internet.  This was because the router was set to a static IP of 192.168.1.1, and was plugged into another router whose gateway was also a static IP of 192.168.1.1 (the sub-sub-network).  (my guess is router hooked up to another router, possibly both of the same brand, both with same defaults).  So I changed the subset for the WRT54G router, giving the WRT54G a static IP address of 192.168.222.1, and assigning addresses 192.168.222.3 to 192.168.222.200 as available for DHCP serving.  However, at that time, while I could ping static IPs on the internet, the DNS wasn’t working right.  Thus, I set up static DNS servers as the default with openDNS servers (208.67.222.222, 208.67.220.220).  Now the router functions as a router.  I changed the network name from “linksys” to “peligro” (we are in a spanish speaking country).  After a few hours I noticed that the locals were popping on by watching the connected machine list, (which is why I increased the number of DHCP addresses available).  Don’t worry, I didn’t change your administrative password, but I did enable logging, so I could see if anybody was abusing the network worse than I was.  I did invest like 20 min in this, so I drew a line in the sand.  I mean, it was hard work balancing the laptop out the window…  And no pringles can around anywhere….  Once I get my internet back online I’ll be out of your hair- if it matters.  If you want to join the wireless network with SSID “gran_gringo” , drop me a line, I owe you one.

[early draft]

I came across this post on Overcoming Bias recently, “Failure by Analogy“.  I would like to take a couple of minutes to discuss tautologies, analogies, projections and logical systems.  First, I’ve never been a fan of neural networks because of their theoretical foundations.  It seemed to me that in my (statistical) community they were a buzzword fad.  I recommend chapter 11 in Elements of Statistical Learning which presents neural networks as a special case of projection pursuit regression if my memory serves me correct.  So I’m critic- what do I suggest if I’m opposed to neural networks?  Linear regression- if that doesn’t satisfy your needs, then we can talk non-linear regression.  That’s my short response.

However, the important issue here is that mathematics is a formal logical system based on tautologies.  That’s all a mathematical proof is to me.  Language is not a logical system.  I admire Wittgenstein for trying to show otherwise.  The fallacy of the argument of analogy is that one is using rhetoric in place of tautologies.

To look at this from another perspective.  There is a real phenomenon here- the brain.  There are many ways to model the brain, and the right one depends on your goal.  You could call these models “projections” of the real phenomenon.  Of course, to do so, I would be entering into an argument by analogy.

FYI: this is a work in progress.  I’m waiting to get copies of the books (that’s going to take 2 months since I’m overseas and I’m not willing to pay an arm and a leg in shipping)…

Feedback for draft 1….

I did read it. It’s definitely the post of yours that’s closest to being a real article — something that could be linked to and passed around online.

I think your basic point is that the mathematical perspective on the phenomenon that Gladwell describes in TP is actually pretty narrow relative to the range of related mathematics.

Are you trying to persuade a layperson fan of TP that the possible mathematical perspective is broader? Or are you trying to persuade a mathematically literate reader of the mathematical point that TP is narrow relative to the rest of the field?

Either way, to improve it I’d suggest

• reading the rest of the book (or at least taking out your mention that you haven’t finished the book)
• changing the title of the blog entry to show that you’re talking about TP rather than the rest of his oeuvre
• showing with actual quotes from the book your mathematical point
• showing how related mathematical phenomena have real-world implications

(draft 1)

As a statistician, I have several contradictory feelings about Malcom Gladwell’s books (Tipping Point, Blink, Outlier).  The main two contradictory feelings I have are, from a mathematical standpoint, the small, tiny topic he discusses, and from a writing standpoint, how well he is able to explain mathematical concepts to his less technical readers.  While I have only read a small portion, I have heard lots of chatter from my peers not trained in mathematics, and read several reviews.  So perhaps it is unfair to write about all of his books.

I read a few chapters of the Tipping Point. First, it is common of me to put down books after reading just a few chapters, so it’s nothing against Mr. Gladwell.  From a mathematical perspective, all I could think about was that the whole book was devoted to one small part of queueing theory, which is one small part of Markov processes, which is one small part of stochastic processes which is one small part of…  The etymology of queueing theory is from queue, the British version of waiting in line- like standing in line at McDonalds.

And I will make an analogy to tetris here, and claim that if he had just spent one chapter explaining one aspect of the popular video game tetris from a mathematical perspective, it would have been much more concise, and a lot less interesting.  I wouldn’t call it pandering to his readers, as he is a writer that wants to be read, and he is teaching the concept.  That’s his objective, to have his readers make it through the book- to present the material from a boring 500+ page operations research book (think Winston here) in a way that keeps the reader excited.  As a writer, that’s the thing that’s really beautiful.

From a mathematical perspective, the whole idea of the tipping point is analogous to tetris in the following way. It’s all about arrivals and departures.  Pieces in tetris, and memes in Tipping Point.  In Tetris, the pieces arrive at a constant rate at each level (the part about different pieces isn’t relevant to this point), and the pieces depart when the player aligns them correctly and the ordered lines disappear. Just as tetris has a limited sized playing area, your mind has a limited number of things it can be thinking about at any given time.  In the Tipping Point, the memes in your mind are like tetris pieces and the tetris playing area.

The tipping point is when the speed of arrival goes from less than to greater than the speed of departure.  Thus, in tetris, the tipping point is when you lose, and all the pieces stack up.  With hush puppies, it’s the time when more people are acquiring the meme for hush puppy acquisition than forgetting about/discarding them.

Mathematical Aside: I’m making a statement that Tipping Point is a special case of Markov processes (stochastic processes).  To be more specific, it is the birth/death part of queueing theory.  In stochastic process speak, the tipping point is when we go from having an arrival rate, (), less than a departure rate, () to the opposite.  That is, the tipping point is the special moment where the system goes from to . This reminds me how I do love a good stationary distribution.

Perhaps my explanation is lacking a bit.  One could extrapolate my argument and make the analogy that playing tetris is like the opposite of being a marketer.   A marketer’s job is to, in the words of the mathematical aside, take the situation where the meme for his product in people’s minds has and make it .

I will say that I greatly enjoyed his story on the moth. (It was free on the podcast, but has since been archived and is now available for sale- I might have a personal copy somewhere to share, though). I’d check his blog for his take on the experience.  The moth is an urban storytelling site in NY, and they record the candid (no notes allowed) stories told by all people.  Their podcasts are usually pretty good, but some can be a little much for virgin ears.

To get back to the contradiction I started with, the really great thing about Mr. Gladwell is his writing.  Obviously – as he tells us in his moth story- he cut his teeth at the Washington Post as a science writer.  And that time, writing about science for laymen – definitely not for Lehmann- really honed his skills.  Perhaps I’m a little jealous that he has this amazing ability to communicate these little pieces of the mathematical trade better than myself.  It’s a little frustrating that being a relative master of one domain in no way affects your ability to communicate that to others.  Mr. Gladwell is an inspiration to me- that I can write about mathematics in a way that engages the reader and has real world consequences (in addition to making large sums of money, of course).  Hopefully the world will use more data and scientific rigor in the making of decisions because of his work.

Brief Description

So my grandparents gave me $1000 when I was 12.  My parents put it in a mutual fund.  When I turned 18 I was faced with the question -after my parents advised me wisely not to blow the money, and after developing a healthy respect for exponential growth- “How should I invest it?”  The classic question.  I played with it.  I watched.  I read books about derivatives, met with day traders, trading strategists, hung with the progeny of hedge fund execs.  I made a fair number of poor decisions when I didn’t have much money.  So when I actually did save money, I knew there were lots of choices.

Fast forward.  I decided short term stuff wasn’t for me- too much money on the line, too much stress, not the kind of lifestyle I wanted.  So, where does one put the money?  I decided to put everything I had in mind together.

Specific Objective

To create a long term (>20 yr) portfolio allocation and rules and implement it on my meager savings.

Product/Results

When I’m 60, ask my about my rate of return.  Oh, and there should be a fair bit of code involved for simulations (back testing if you use the finance lingo).

Status

Should be farther ahead than it is.  Basically, a while ago I read Hulls Options, Futures and Other Derivatives.  Recently I started Swenson’s Unconventional Success.  I’ve organized some tools I found.

Current Research

It all started with a bunch of thought experiments and some brief looks at historical data.  My basic idea is to “optionize” the Swenson portfolio.  What I mean by this is as follows with a covered call example.

Imagine you have $100 you want to invest in the S&P.  A typical strategy would be to go long $100 in an S&P ETF.  While historically you would be getting in the 5-10% return in inflation corrected dollars, if you had done this a year ago, you would have lost big- like up to 45% (using the 52 week high/lows).  However, if, instead of going long, you had instead bought a call option for a $5 fee on $95 worth of S&P ETF, you would have the same basic return profile if the ETF went up.  However, by purchasing a call, you are protected if the price falls.  If you had taken that $95 that covered the call and put it in government bonds or whatever, you wouldn’t have lost anything due to the S&P.

Basically, you could set up little hedges, and roll the hedges forward.  I think those were the terms that Hull used.  I think you need economies of scale to really get this going, but when the market is bubbling, it seems like the strategy to use.  Now that the market might actually be reasonable priced, well, let’s see what the simulations say.

Some open questions I have are:

• The whole portfolio balancing thing relies on having uncorrelated baskets
  • How do I find the best uncorrelated baskets?
  • I was going to check to make sure that they weren’t cointegrated, but that’s about as far as I’ve gotten without actually touching data.
  • Would using NCAIS data be helpful or just get in the way?
• What’s the best way to use historical data
  • What matters is the return with respect to a particular currency at a particular time.
  • As far as running simulations/backtesting, what time chunks should I be working with?
  • Which particular values should I be worried about?  Capital gains tax rates? Federal funds rate? Ted spread? Treasury bonds? Volatility? Inflation? The distributions probably aren’t independent.
  • What does historical data tell us about different methods of choosing uncorrelated baskets?
• What data should I be using?
  • Daily? Monthly? Yearly?

Brief Description

Disk space, memory and time are increasingly common constraints in applied statistics. The size of
datasets is growing at a fast pace, yet the computational tools needed for this type of analysis are not
growing at a sufficient pace. This document is an attempt to describe the incorporation of 3 ideas to
improve this situation: 1) Having a standard language for describing a statistical model, as in Ch 2 of
“Statistical Models in S” 2) Having a fast implementation (eg compiled and not ineractive) of the S
language which 3) uses ScaLAPACK to create a layer of transparency to problem size. Thus, a user
should be concerned with creating a statistical model and analyzing its output, not having to worry
about the computational platform and details underlying its implementation.

Specific Objective

It was to try and see if there was any interest in brute force tools for large datasets.

Product/Results

Only a brief white paper.

Status

Out, presumed dead.  See postmortem.

Postmortem

Well, at the time I didn’t have much experience with big databases.  There was pretty much no interest.  In my own perspecitve, I came to see this as a brute force approach to the issue, and found the bootstrap to be much more elegant and applicable here.  Further, I several failures trying to write SQL to do matrix operations.  It’s possible that there may be some OLAP solutions which are more elegant, but I don’t know much about that.  I did find some research about doing statistics on massive SQL datasets that seemed a little more promising.

Further, a lot of these tools were developed for other communities (see PDE folks), who don’t currently have any elegant tricks for getting around things like Navier-Stokes.  I think.  Bummer for them.  But at least they actually have a valid excuse for working on big parallel machines.

One thing I’ve wanted for a while is a piece of software to diagram sentences for me.  After a mildly exhaustive search, and several emails the Stanford Natural Language Parser is about as good as I could find.  I know natural language is an excruciatingly difficult field, and I know that I wouldn’t be able to do anything without standing on the shoulders of giants.  I think it was the work of Pennebaker that recently got me thinking about this.  As well as a discussion with my advisor about quantative research where an interview is the data.  The question being “What tools do we have for analyzing text?”