downloading folders from google drive.

I wanted to download some course material on RL shared by the author via Google drive using the command line.  I got a bunch of stuff using wget a folder in google drive was a challenge. I looked it up in SO which gave me a hint but no solution. I installed gdown using pip and then used: gdown --folder --continue https://drive.google.com/drive/folders/1V9jAShWpccLvByv5S1DuOzo6GVvzd4LV if there are more than 50 files you need to use --remaining-ok and only get the first 50. In such a case its best to download using the folder using the UI and decompress locally. Decompressing from the command line created errors related to unicode but using the mac UI I decompressed without a glitch.
Post a Comment

Business analytics - P2P loan data

Analyzing an admixture of categorical and numerical variables.

Trellis chart of loan age distributions faceted by
clients' credit levels & loan status.
This time I look at visualizing p2p loan data which is mostly categorical in nature (factors rather than numeric). While reproducing the charts in the book was somewhat challenging, I also went a step further and revealed some expected patterns as well as some surprising deviations.

First I created the density plots. This was straight forward in R. I did decide to transform one of the variables with a log function. Since it had zero values those rows were dropped. So I added a minimal increment to all the values and got them back in. I had thought the graphs were not so elucidating so I explored them further and when I faceted them further into trellis graphs (by credit ratings) some interesting patterns started to emerge.

These charts provide a high grain view of the data. One can easily interprets the interplay of 4 variables. Which are not visible in the aggregate views provided in the text.

As might be expected at the first maximum of the Loan Distribution we see the the beginning of the Default maximum. This would come about because before defaulting the loan taker would be maximally late.

What is perhaps surprising is that at some credit B and D levels there are two peaks in the  distribution of Defaulting while at the others exhibit a single peak

However since the code loads, installs and saves - it is not so easy to read.

Ggplot2 is lacking the ability to generate a Spineplot, though I provide a link showing how these might be created one rectangle at a time. I showed a couple of close hits but no success - this comprises the contents of the first file bellow. While researching this I leaned a bit about ggplot2 stats feature and I found that spineplots have been commented out of ggplot2's source.

Once I gave up on ggplot I quickly got good the required chart. R has a simple command to generate a Spineplot. I am skeptical as to the effectiveness of a Spineplot is as a visualization option for  information.


Further reading

  • ggplot2 Mosaic Charts

Comments

Post a Comment

Popular posts from this blog

Moodle <=< Mediawiki SUL integration - first thoughts

Image
SUL is  Wikipedia's  Single User Login system The goal is to use it to authenticate moodle users there is one caveat - what happen if  the moodle user has no account. Moodle and MediaWiki account creation require different information. Since we'd like to use MediaWiki's standard which is highly permissive. It is necessary to change  moodle's requirements. Challenges Moodle does not explicitly define an object for setting registration requirements.  This needs to be accomplished by admins via edit a number of files manually.  errors will place the users in a limbo/blocked state... This is implemented differently on different version of moodle. This will break if the changes are overwritten by a system is upgrade. Directions Ideally Moodle should have a registration policy  object which allows the admin to define which fields are required and if they need to be unique . For example Moodle complains if different users share an email adr
Read more

downloading folders from google drive.

I wanted to download some course material on RL shared by the author via Google drive using the command line.  I got a bunch of stuff using wget a folder in google drive was a challenge. I looked it up in SO which gave me a hint but no solution. I installed gdown using pip and then used: gdown --folder --continue https://drive.google.com/drive/folders/1V9jAShWpccLvByv5S1DuOzo6GVvzd4LV if there are more than 50 files you need to use --remaining-ok and only get the first 50. In such a case its best to download using the folder using the UI and decompress locally. Decompressing from the command line created errors related to unicode but using the mac UI I decompressed without a glitch.
Read more

Insight into progressive web apps

Image
Some notes from a Meetup on PWAs in January 2016. I feel quite knowledgable on PWA but I wanted to learn more on implementing service worker. I ended up adding some research and collecting some great resources. However I ended up getting more detailed materials on the service worker based on google's developers docs. Also the resources have been expanded. https://www.meetup.com/The-Future-is-Javascript/events/246346861/ Service worker Service Workers Its just a simple JavaScript file that sits between you and the network – It runs in another thread – It has no access to DOM – It intercepts every network request (including cross domain) Entry point: self.caches (in service worker) or window.caches (on page)   Registering a Service Worker • Works with promises • Re-registration works fine In main.js navigator.serviceWorker.register('/sw.js').then(function(reg){ console.log('regsitered'); }.cat
Read more