Posts

Showing posts with the label SQL

downloading folders from google drive.

I wanted to download some course material on RL shared by the author via Google drive using the command line.  I got a bunch of stuff using wget a folder in google drive was a challenge. I looked it up in SO which gave me a hint but no solution. I installed gdown using pip and then used: gdown --folder --continue https://drive.google.com/drive/folders/1V9jAShWpccLvByv5S1DuOzo6GVvzd4LV if there are more than 50 files you need to use --remaining-ok and only get the first 50. In such a case its best to download using the folder using the UI and decompress locally. Decompressing from the command line created errors related to unicode but using the mac UI I decompressed without a glitch.

SQL Dojo

Image
TLDR: Imagine just before your DS interview - you are NEO your coach is Morpheus, and you will be practice SQL in rapidly changing schemas. Now here is a little project I thought up: Despite any number of excellent SQL based projects I have created I tend to get rusty in SQL as I don't use it on a regular basis. I decided it might be worthwhile  to setup a virtual space to practice, hence the dojo. The dojo lets a student practice analytical sql primarily queries analysts use. Ultimately I'd like to to use it in an agile manner as an LMS with a minimal UI.  This would require creating a story for each query and a test that the query returns a good answer. Also to make things interesting the tasks should be related and proceed from easy to more challenging and cover a number of techniques like filtering, aggregation and subqueries. However, initially I want to have things up and running quickly and to collect questions and answers that reflect how to do create view...

Finding your friends on Wikipedia

Image
A segment of a social network (Photo credit: Wikipedia ) In my opinion, the most notable missing features of Wikipedia is the ability to network socially . Lack of social features conspire to minimise participants'  social capital (the expected collective or economic benefits derived from the preferential treatment and cooperation between individuals and groups). Lack of networking capabilities is in line with a pervasive attitude that ownership of contributions should be suppressed, external authority ignored and that knowledge matters in the project. Accordingly social costs of participation are growing. Participants in the project have great difficulty in acquiring social capital. The stakes of establishing world opinion tend to outweigh social stakes. Accordingly there is a growing gender gap, under-representation of cultural minorities and disrespect of real world authority. Currently users will try to establish social capital by listing on their user page their m...

Tips on working with SQL Fiddle

Image
C-3PO vs. Data (137/365) (Photo credit: JD Hancock ) Trying to get help with SQL ?  Chances are that you could use the open-source web application  mysqlfiddle.  Usually questions asked on the web leave much to be desired in terms of precision. The good news is that with Sqlfiddle you can set up a copy of the relevant sql schema, add some sample data,  add your best attempt and and share these with the people you are chatting or corresponding with. Next follows an example environment I set up to explore on collecting editor sessionstatistics for wikipedia's undocumented revision_user table. Working with a full environment was not practical. Working with the sqlfiddle actually ended up as a big time saver since testing the queries on a live database would have taken hours. http://sqlfiddle.com/#!2/d2f02/22 It sometimes seems that setting up a Database environment is a bit of an overkill - especially when it is just to ask a question. However if you have a wor...

Mediawiki User edits by Namespace

Image
\ During work on Orwell01 I got to look through the code of X! counter tool on the WMF toolserver. Mediawiki's API does not expose a method to get a user's edit counts per namespace. When I was chatting with some WMF developer at the Amsterdam Hackathon - I was told that this was not exposed since it was not stored in the database and had to be worked out in an application. And indeed there is a popular edit counter called X! edit counter maintained by User:TParis that does just this. However this edit counter does not expose the namespace stats either in its undocumented API either.... Looking at the code I started to explore the database queries being done and found the following:

Popular posts from this blog

downloading folders from google drive.

Random Thoughts on Linear Regressions

SQL - Selection v.s. Projection