Bayesian.Ninja

Teaching History ? One of the requirements for the historical courseware project is to add a timeline for each unit. A time line allows a teacher to capture   the order of events in a way that students can visualize. While events can be ordered in time, they can usually be located in space as well. To account for this duality timelines can be be paired of with a map as well. Putting the two together and we can get interactive animated maps. But this is not the end either. Events on a Time lines may be clustered to indicate types of historical patterns. Last but not least a timeline can be enhanced with some kind of thematic data which is associated with the events. If the events are appropriately tagged then the time line can be faceted accordingly. Date presented in this fashion becomes a powerful dynamic visualization tool more so it it can be made  interactive. Bellow is Charles Minard 's innovative  chart of Napoleon's ...

Last week I was asked by +Charles Matthews  to review WM.UK upcoming Wikipedia tutoring system being developed on on the open source Moodle system. I've been thinking of different option to better support Wikipedia coaching in different languages  I have been reviewing and testing Moodle in the last few weeks to that end. I'm really interested in developing course ware for both English and Hebrew editions (as a start). Multilingual content While Moddle is internationalized, support of multilingual content requires using a somewhat clumsy technique. One need to use html and store different languages content in top level div elements. Then you have to switch on Moodle's multilingual filter which can hide all the language version which are incompatible with the language selector's settings. However due to a bug the the html editor mangles the html code and mangles the top level <span...

Social Network Analysis or (SNA) is one of the most powerful tools available in the Social Sciences today. It allows to identify key players in a system, find structural holes and many other aspects of a society which are less accessible without the network paradigm. Gephi is an opensource package written in Java which is aimed at performing SNA. I came upon this in the recommendation of the new Coursera course on SNA . Gephi was selected due to its many features, because it is free software and since it cross platform. Another of it its advantages is in visualization. Here is an image of a classical data-set showing the dining school partners. This is a directed graph with 26 vertecies and 52 nodes indicating each girl's top two preferences for a dinner partner. The data was collected circa 1905 and published in the 1950. The outgoing degree is therefore 2 but the incoming degree varies from 1 to 6 showing that some girls (Eva an...

Analyzing an admixture of categorical and numerical variables. Trellis chart of loan age distributions faceted by clients' credit levels & loan status. This time I look at visualizing p2p loan data which is mostly categorical in nature (factors rather than numeric). While reproducing the charts in the book was somewhat challenging, I also went a step further and revealed some expected patterns as well as some surprising deviations. First I created the density plots. This was straight forward in R. I did decide to transform one of the variables with a log function. Since it had zero values those rows were dropped. So I added a minimal increment to all the values and got them back in. I had thought the graphs were not so elucidating so I explored them further and when I faceted them further into trellis graphs (by credit ratings) some interesting patterns started to emerge. These charts provide a high grain view of the data. One can e...

In this section the challenge is to explore data comparing digital and traditional media options. The digital media is priced differently from the print. I think that this is not the clearest visualization in the world. But it made for a great learning experience. To create the geographic plot I used the map package together with ggplot2  which is based on by + Leland Wilkinson's  book The Grammar of Graphics . I had read the book in 2007, but found it disappointing that the system had been used only in a propitiatory format. However ggplot2  brought this powerful idea into the opensource world or R. After researching a number of options I eventually decided using ggplot2 . I preferred it since it ended up reducing the code to simple and clean form. As you can see I've improved the documentation format to include loading and installation of required packages. Geo-Tagged sales (digital are in red paper are black) I...

Working with Soft drinks sales data (Section 2.3) The images produced to analyze the time series are not standard R graphics. As such I needed to dig a little deeper to replicate them. The were two tricks required to replicate this, How to draw one plot over another. How to use the same axis definition for all three plots. (these are hard coded). While researching this I noticed that thi=s example does not even scratch the surface of R's capabilities for handling time series. However perhaps these are covered later. Anyhow here is the image As explained the sales figure can be seen to have a periodic element and growth element - possibly exponential. Plotting the simpler unenhanced plot side by sided requires the layout command. I may do this later, but now it is time to move on. Further reading Data galore: Time series Data Library R as a Tool in Computational Finance by +John P. Nolan a little book of R for time series  by  ...

Working with Direct marketing data (Section 2.2) This section discusses exploring data using data transformation, trellis graphs and using a scatter-plot matrix  explore direct marketing data. My own experience doing direct marketing campaigns for real estate was that we did not have such data-sets to work with - just lists of leads based on internet based research which yielded extensive data sets but of a sparser nature. Contacting the Postal authority did offer to us a service where based on a confidential dataset they allowed clients to send direct mail to clients according to some many criteria. However they could not provide the size of samples in real time nor did they provide the addresses - only the ability to send them letters for a fee. However their dataset was stale (3+ years old at best and they were going to renew it only in six months). Since the data was old and since there would be no way to check the reliability of their service we p...

Working with real estate data (Section 2.1) This section discusses exploring data using summary statistics, histograms and scatter-plots to better understand the data. Some of these tasks can be done using excel but it probably requires more work Most  of the Material in the sections 2.2 and later cannot be done using excel and requires a more powerful data handling  package like R or the commercial packages like SPSS, SAS, Matlab etc. Loading, viewing and summary statistics Generate histograms Creating a box plots for the prices etc.: Creating a scatter plot for the prices against apartment size: I used this webiste to get these samples together. I also work with The R book by +Michael J. Crawley 

Wolfgang Jank's text  "Business Analytics for Managers" is part of the Use R! series published in 2011 by Springer. It stand out as easy introduction for using R for exploratory data analytic and data modeling at an introductory level. While R and statistics are skill not easily acquired  working with the book makes for an easy learning curve by focusing on the business side of the work while omitting much of the platform specific implementation details. This makes some sense if the manager will want to ask someone else to work with R to get him the results. The text explains how to work with different type of data and how a manager would  analyse the different data-sets. It explains the benefits of using a large cross section of R's visualization techniques for assessing unfamiliar data for global or cross sectional patterns. It then goes on to explain the...

Due to a kind grant by the WMF foundation I was able to attend Wikisym 2012 in  Linz  Austria what follows is my report on the event. Background English: The renovated Ars Electronica Center at Linz, seen from the bridge across the Danube at night (Photo credit:  Wikipedia ) I am a Wikipedian based in Budapest Hungary. I have been active for the last year with WM.HU and participated in a number of the local  chapter's events ever since being introduced to them at Wikimania 2012 in Haifa. During such a meeting I, Bence Damkos, and other chapter luminaries got to discussing many apparent cultural paradoxes taking place in the virtual community. Since I was studying the theory of games at the time I began to notice that some of the situations were very similar to a classic game such as the prisoner's dilemma and the battle of the sexes while others resembled second-price sealed actions and bargaining games. I was intrigued an...