downloading folders from google drive.

I wanted to download some course material on RL shared by the author via Google drive using the command line.  I got a bunch of stuff using wget a folder in google drive was a challenge. I looked it up in SO which gave me a hint but no solution. I installed gdown using pip and then used: gdown --folder --continue https://drive.google.com/drive/folders/1V9jAShWpccLvByv5S1DuOzo6GVvzd4LV if there are more than 50 files you need to use --remaining-ok and only get the first 50. In such a case its best to download using the folder using the UI and decompress locally. Decompressing from the command line created errors related to unicode but using the mac UI I decompressed without a glitch.
Post a Comment

Business analytics - plotting Geo-tagged data

In this section the challenge is to explore data comparing digital and traditional media options. The digital media is priced differently from the print. I think that this is not the clearest visualization in the world. But it made for a great learning experience.

To create the geographic plot I used the map package together with ggplot2 which is based on by +Leland Wilkinson's book The Grammar of Graphics. I had read the book in 2007, but found it disappointing that the system had been used only in a propitiatory format. However ggplot2 brought this powerful idea into the opensource world or R.

After researching a number of options I eventually decided using ggplot2. I preferred it since it ended up reducing the code to simple and clean form. As you can see I've improved the documentation format to include loading and installation of required packages.

Geo-Tagged sales (digital are in red paper are black)


I ended up with a small annoyance - the center of the plotted points contain a block dot which makes this look like an the results of an military exercise.

ggplot is very intuitive but does require a couple of hours to figure out. To get you started here are  a couple of articles and books which I found noteworthy of mention.
  • The ggplot tutorial I used to get started.
  • ggplot documentation I used to fix the details.
  • If R is missing your map it is possible to get more via OpenStreetMap as shown here.
  • An advanced map graphic inspired by the Financial Times 2004 election results from here, though it is not very clear what is really going on there.
  • There is a book covering this library - ggplot2: Elegant Graphics for Data Analysis by +Hadley Wickham
  • Finally the text "R Graphics" by +Paul Murrell (2006) covers using the maps package without ggplot.

Comments

Post a Comment

Popular posts from this blog

Moodle <=< Mediawiki SUL integration - first thoughts

Image
SUL is  Wikipedia's  Single User Login system The goal is to use it to authenticate moodle users there is one caveat - what happen if  the moodle user has no account. Moodle and MediaWiki account creation require different information. Since we'd like to use MediaWiki's standard which is highly permissive. It is necessary to change  moodle's requirements. Challenges Moodle does not explicitly define an object for setting registration requirements.  This needs to be accomplished by admins via edit a number of files manually.  errors will place the users in a limbo/blocked state... This is implemented differently on different version of moodle. This will break if the changes are overwritten by a system is upgrade. Directions Ideally Moodle should have a registration policy  object which allows the admin to define which fields are required and if they need to be unique . For example Moodle complains if different users share an email adr
Read more

downloading folders from google drive.

I wanted to download some course material on RL shared by the author via Google drive using the command line.  I got a bunch of stuff using wget a folder in google drive was a challenge. I looked it up in SO which gave me a hint but no solution. I installed gdown using pip and then used: gdown --folder --continue https://drive.google.com/drive/folders/1V9jAShWpccLvByv5S1DuOzo6GVvzd4LV if there are more than 50 files you need to use --remaining-ok and only get the first 50. In such a case its best to download using the folder using the UI and decompress locally. Decompressing from the command line created errors related to unicode but using the mac UI I decompressed without a glitch.
Read more

AWS CloudFormation Pros and Cons

AWS CloudFormation Pros and Cons So I'm building a PAAS product that does ML based optimisations and that means doing work in the cloud. The ML is a neat feature but without the basic product nothing will happen and to bootstrap this project on AWS I tried to make use of CloudFormation a service that automates creation and destruction of service stacks. Based on a week's worth of experimenting with CloudFormation I will try to answer the question: "Is learning CloudFormation worth the effort?" Despite the rant CloudFormation support creation, updating and deletion of entire stacks of services. SAM is built on top of CloudFormation and It has a visual editor. The way CloudFormation is described, is that you can copy paste snippets to create resources and build a library of reusable components. This is a simplistic point of view. In reality you need to bring properties, specify dependencies, and introduce signalling mechanisms to ensure your template works. T
Read more