Posts

Showing posts from January, 2020

downloading folders from google drive.

I wanted to download some course material on RL shared by the author via Google drive using the command line.  I got a bunch of stuff using wget a folder in google drive was a challenge. I looked it up in SO which gave me a hint but no solution. I installed gdown using pip and then used: gdown --folder --continue https://drive.google.com/drive/folders/1V9jAShWpccLvByv5S1DuOzo6GVvzd4LV if there are more than 50 files you need to use --remaining-ok and only get the first 50. In such a case its best to download using the folder using the UI and decompress locally. Decompressing from the command line created errors related to unicode but using the mac UI I decompressed without a glitch.
Post a Comment
Read more

SQL Dojo

Image
TLDR: Imagine just before your DS interview - you are NEO your coach is Morpheus, and you will be practice SQL in rapidly changing schemas. Now here is a little project I thought up: Despite any number of excellent SQL based projects I have created I tend to get rusty in SQL as I don't use it on a regular basis. I decided it might be worthwhile  to setup a virtual space to practice, hence the dojo. The dojo lets a student practice analytical sql primarily queries analysts use. Ultimately I'd like to to use it in an agile manner as an LMS with a minimal UI.  This would require creating a story for each query and a test that the query returns a good answer. Also to make things interesting the tasks should be related and proceed from easy to more challenging and cover a number of techniques like filtering, aggregation and subqueries. However, initially I want to have things up and running quickly and to collect questions and answers that reflect how to do create view...
Post a Comment
Read more

Popular posts from this blog

Moodle <=< Mediawiki SUL integration - first thoughts

Image
SUL is  Wikipedia's  Single User Login system The goal is to use it to authenticate moodle users there is one caveat - what happen if  the moodle user has no account. Moodle and MediaWiki account creation require different information. Since we'd like to use MediaWiki's standard which is highly permissive. It is necessary to change  moodle's requirements. Challenges Moodle does not explicitly define an object for setting registration requirements.  This needs to be accomplished by admins via edit a number of files manually.  errors will place the users in a limbo/blocked state... This is implemented differently on different version of moodle. This will break if the changes are overwritten by a system is upgrade. Directions Ideally Moodle should have a registration policy  object which allows the admin to define which fields are required and if they nee...
Read more

Random Thoughts on Linear Regressions

Regression Analysis TLDR Regression is the oldest and most powerful tool in a data scientist's toolbox. Under ideal conditions multiple linear regression would be the best and only tool a data scientist would want to use... In reality you would use a modern nonparametric variant or a different algorithm. Still, the main ideas I . discuss here  will pop up in many other models and algorithms. This post is my brain dump on regression - I'll update it as time allows to cover the many aspects of this technique.
Read more

Big Data Analytics Israel - New Year, New Data Scientist Job: 5 Things To Think About

Image
Data science interviews can be over whelming  New Year, New Data Scientist Job: 5 Things To Think About My notes: https://www.meetup.com/Big-Data-Analytics-Israel/events/253124286/ The first talk was by: Raya Belinsky -  "New job - yes or no?" The talk about finding your next job or reinventing your current jobs. Miss Belinsky's humour and background as an executive life-coach made this talk both pleasant and worth-while. She covered her operational definition of job burnout Linkin profile - complete the profile (it tells you what to do) The CV - ask 2 people to prepare it The Interview - e.g. prepare 3 questions Each had at least a couple of points worth taking care of in your next round of job search. Check out the talk and slides when they go online. Second talks by: Nathaniel Shimoni - "Life story" Mr Shimoni is an experienced story and had a compelling story to tell and his own twiting path to  becoming  a da...
Read more