State of Data #102
June 7, 2012
#1Read_this_Week – ‘The Girl with the Dragon Tattoo’ makes a SQL mistake – ‘My favourite part, though, was when Lisbeth Salander begins to solve a 40 year old murder cold case using SQL.’
#analysis – A good series on ‘Moving Beyond Conversion Rates’ –
#big_data – DataPop’s story – startup that ‘relies on semantic search and natural-language processing to infer connections between what consumers enter into the search window and what they really want, and then on machine learning to help with everything from determining common spelling mistakes to search construction to the sequence of events that leads to a purchase’
#Data_Science – How to predict Eurovision Song Contest winners
“1. Set based processing will likely be much faster than row based processing. Our experiment of processing 100K rows showed row based processing was 3700 times slower than set based processing. Not twice as slower or even 10 times slower… 3700 times slower!
2. Compared to set based processing, row based processing times degrade much quicker than set based processing. That is, row based processing does not scale nearly as well as set based processing. We saw this in that the linear trend line for row based processing was 0.00259 compared to 0.00000 for set based processing”
#idea – Nassim Taleb, author of bestseller ‘Black Swan’, writes about ‘side effects’ of too much reliance on data –
‘In business and economic decision-making, data causes severe side effects —data is now plentiful thanks to connectivity; and the share of spuriousness in the data increases as one gets more immersed into it. A not well discussed property of data: it is toxic in large quantities —even in moderate quantities.’
#visualization – “What’s wrong U.S.” is visualization of Target and Walgreens’ sales data to see which state has most headache problems or allergy or stuffy nose
- FourSquare data transforms to Jewellery
- Data Visualization Sculptures (‘musical Score, translating weather data of a particularly nasty winter storm in 2007 and the memories it evoked of the Blizzard of 1978’)
- Do children with bigger feet really spell better? Yes, because they’re older!
- Venn pie-agram – Winner of Bad Graph Contest