Posts

Showing posts with the label devops

downloading folders from google drive.

I wanted to download some course material on RL shared by the author via Google drive using the command line.  I got a bunch of stuff using wget a folder in google drive was a challenge. I looked it up in SO which gave me a hint but no solution. I installed gdown using pip and then used: gdown --folder --continue https://drive.google.com/drive/folders/1V9jAShWpccLvByv5S1DuOzo6GVvzd4LV if there are more than 50 files you need to use --remaining-ok and only get the first 50. In such a case its best to download using the folder using the UI and decompress locally. Decompressing from the command line created errors related to unicode but using the mac UI I decompressed without a glitch.
Post a Comment
Read more

AWS CloudFormation Pros and Cons

AWS CloudFormation Pros and Cons So I'm building a PAAS product that does ML based optimisations and that means doing work in the cloud. The ML is a neat feature but without the basic product nothing will happen and to bootstrap this project on AWS I tried to make use of CloudFormation a service that automates creation and destruction of service stacks. Based on a week's worth of experimenting with CloudFormation I will try to answer the question: "Is learning CloudFormation worth the effort?" Despite the rant CloudFormation support creation, updating and deletion of entire stacks of services. SAM is built on top of CloudFormation and It has a visual editor. The way CloudFormation is described, is that you can copy paste snippets to create resources and build a library of reusable components. This is a simplistic point of view. In reality you need to bring properties, specify dependencies, and introduce signalling mechanisms to ensure your template works. T...
Post a Comment
Read more

The age of Selenium

Image
Some quick notes from the 1 0th selenium meetup which took place at the IronSource offices. My interests were primarily for testing websites and the spacial use case involving "analytics" setups.  It turns out that both simple and complicated analytics setups require about six testing sessions. When working with large web sites analytics setup tend to get broken routinely. While custom alerts can help to detect large disasters, a automated testing tool is invaluable for tracking these type of failures early. Oxygen IDE The bleated Selenium IDE  (Photo credit: Wikipedia ) The meetup started with an unscheduled  lightning talk by Nachum Dimer. This, for me, was the most productive talk, quite possibly because it was just 10 minutes and so had no fluf. It covered Oxygen Ide . Oxygen is a new Selenium framework and integrated development environment. Oxygen IDE allows recording on Internet Explorer, Firefox and Chrome. which is a significa...
Post a Comment
Location: Lilienblum Street 22, Tel Aviv-Yafo, Israel
Read more

Popular posts from this blog

downloading folders from google drive.

I wanted to download some course material on RL shared by the author via Google drive using the command line.  I got a bunch of stuff using wget a folder in google drive was a challenge. I looked it up in SO which gave me a hint but no solution. I installed gdown using pip and then used: gdown --folder --continue https://drive.google.com/drive/folders/1V9jAShWpccLvByv5S1DuOzo6GVvzd4LV if there are more than 50 files you need to use --remaining-ok and only get the first 50. In such a case its best to download using the folder using the UI and decompress locally. Decompressing from the command line created errors related to unicode but using the mac UI I decompressed without a glitch.
Read more

Random Thoughts on Linear Regressions

Regression Analysis TLDR Regression is the oldest and most powerful tool in a data scientist's toolbox. Under ideal conditions multiple linear regression would be the best and only tool a data scientist would want to use... In reality you would use a modern nonparametric variant or a different algorithm. Still, the main ideas I . discuss here  will pop up in many other models and algorithms. This post is my brain dump on regression - I'll update it as time allows to cover the many aspects of this technique.
Read more

SQL - Selection v.s. Projection

Selection and projection are two high level processes taking place when SQL queries are executed. Selection is choosing some records (rows) from a table and leaving others out. e.g. rows having name='oren'. Projection is the choosing some columns from each record and leaving others out. e.g. only name. So the select keyword performs projection while the where and keyword performs selection. Clearly the choice of using the keyword select for projection (choosing columns) rather than choosing rows, is an unfortunate flaw in the design of SQL, but this oversight is too well established to be fixed.
Read more