downloading folders from google drive.

I wanted to download some course material on RL shared by the author via Google drive using the command line.  I got a bunch of stuff using wget a folder in google drive was a challenge. I looked it up in SO which gave me a hint but no solution. I installed gdown using pip and then used: gdown --folder --continue if there are more than 50 files you need to use --remaining-ok and only get the first 50. In such a case its best to download using the folder using the UI and decompress locally. Decompressing from the command line created errors related to unicode but using the mac UI I decompressed without a glitch.

Automating web app development with Polymer and Yeoman

Yeoman lets you configure and
 stamp out sophisticated boilerplate
projects from the command line.
In a digital marketing agency  the data science team may be asked to provide for each client's campaign fairly similar  media reports and explanatory analytics dashboards for both external clients and internal clients. For longer term project we may be asked to also provide predictive analytics. The data comes from advertisers Google, Facebook, Taboola, OutBrain, phone tracking metrics API, outcomes are channels via segment into say Google Analytics which has both an API and polymer components. Usually there will be additional data science products prediction special segment data, funnels, market research, attribution charts which and long term data in BigQuery which has an API as well. Some vendors don't have an API so to access their data it is exported into a CSV and placed into google sheets which has an API and a Polymer component phone tracking. It takes too much work and time to code all these dashboards unless the campaign is long term. But when one uses a generator like Yeoman and incrementally add each part the creation of much of these dashboards can be automated reducing time and complexity for creating these solutions.

A second use case for using this combo is creating html5 banners. Often campaigns require creating many simail banners especially if you want to scientifically optimise your creative using a fractional factorial experimental design. Here is the Buildabanner Yeoman generator


Polymer together with Yeoman can help  kick start new web project with an opinionated fully baked tooling infrastructure. Each new edition of polymer has many  changes and different tooling.  Yeoman is not very well documented and challenging to integrate into the increasingly automated build formats most CLI use today. So while these have fairly steep learning curves which may make it difficult to justify  the return on time invested for tooling etc. But together if you have many similar projects planned, or are building a self-serve system Yeoman and polymer may be just the right fit.

Yeoman is an automation tool for creation of a web project. The more structured your projects are the more Yeoman is going to save you time. It also shares with web components the notion of compatibility which can help support complexity. However to make a Yeoman generator bullet-proof may require long term support and fixing bugs which occur on other people's systems.

The two have been combined in the Polymer CLI, though currently you may be interested in the following resources if this is a project you wish to automate. It also allows teams to concentrate knowledge into a generator which will more readily supports additional automation via scripts build tools and a split production and development pipeline as well as CI down the line.

Another issue common to work with large boiler plate projects - few people know what all the boiler plate is doing, how it can be tested or changed. So consider that you should document the project thoroughly.

Resources on accelerating Polymer projects with Yeomen

I cover these because Yeoman is easy to get started with but you soon end up interacting with Bower, NPM as well as other tools which boils down to a steep learning curve.

The Polymer the Javascript library that teaching the browser new tricks. Polymer has had strong ties to Yeoman project - perhaps since the polymath Addy Osmani is on both development teams. Yeoman is described as "The web's scaffolding tool for modern web apps".

Here is an article from 2013 on creating polymer projects using Yeoman, a year later he release the following video:

+Addy Osmani -  Building a Polymer app with Yeoman 2014

YOLOmer! Polymer and Yeoman for lighting fast dev

When Polymer 1.0 was released it was introduced with a couple of  starter  kit projects one for new users and another for power users. This project included some fairly sophisticated use of tooling to provide a plethora of configurable features such as offline support using service workers. At the polymer summit in 2015. Rob Dobson introduced a polymer Yeoman generator that stamps out a psk project

+Rob Dobson - End to End with Polymer from The Polymer Summit 2015

In 2016 polymer introduced routing and layouts components and behaviors. These were released with a Polymer CLI a command line tool based on Yeoman that provides unified install and  access to many tools that a used with Polymer. However, the PSK2 project was not updated to work with these and eventually a much simpler starter project was recommended.

These were introduced by Rob Dobson in 2016 in a couple of Polycast episodes 52 and 53 focused on the PSK2.

Rob Dobson - How to build a CLI generator -- Polycasts #53

Polymer CLI Generators 101 

Polymer CLI allows us to generate components. There are many types of components and creating a working environment with a demo and tests takes lots of work and research. We are also at the cusp between polymer 1 and polymer 2 where es2015 rules supreme.

So what Let's build some generators for custom components in Polymer. This is one use case where working with generators can have a significant payoff in time savings

  • Polymer 2 preview elements
  • es6 polymer element element 
  • psk2 with es6 support
  • an element in psk2 with es6 support
  • a style element
  • a behaviour
  • d3 element etc (integration with libraries that can play with polymer) What are the requirements for a lib so it can be integrated with polymer's data binding model.
  • a PWS project with firebase support

Additional References


Popular posts from this blog

Moodle <=< Mediawiki SUL integration - first thoughts

downloading folders from google drive.

AWS CloudFormation Pros and Cons