Just finished another Chrome extension
Clippy is a Chrome extension which allows users to tag any url in a web page so we can later recognize them.
You may read more about it here Clippy
Object attribute lookup in Python
Resources
descriptors
- https://docs.python.org/2/howto/descriptor.html
- http://stackoverflow.com/questions/3798835/understanding-get-and-set-and-python-descriptors
python 2 docs
- https://docs.python.org/2/reference/datamodel.html?highlight=metaclass
great insights
- http://blog.ionelmc.ro/2015/02/09/understanding-python-metaclasses/
__mro__
(Method Resolution Order)
- http://stackoverflow.com/questions/2010692/what-does-mro-do-in-python
- http://stackoverflow.com/questions/1848474/method-resolution-order-mro-in-new-style-python-classes
simulate a __getattribute__
call
- http://codereview.stackexchange.com/questions/92497/simulate-object-getattribute-in-python
Definitions
- Everything in Python is an object (i.e. classes, modules, the numbers, the strings, etc)
- A
class
is also anobject
- Every
object
is an instance of aclass
(example: isinstance(5, int) ) - Because of that, every
class
is an instance of a special kind of class calledmetaclass
- An instance is created by calling a class object
- A non-data descriptor is an object following the data descriptor protocol as described in the docs
- A data descriptor is a descriptor which defined both the
__set__
AND__get__
methods __mro__
is a tuple of classes that are considered when looking for base classes during method resolution
Code snippet
Instance attribute look up
The implementation works through a precedence chain that gives data descriptors priority over instance variables, instance variables priority over non-data descriptors, and assigns lowest priority to getattr() if provided.
Given a Class “C” and an Instance “c” where “c = C(…)”, calling “c.name” means looking up an Attribute “name” on the Instance “c” like this:
- Get the Class from Instance
- Call the Class’s special method
__getattribute__
. All objects have a default__getattribute__
Inside __getattribute__
- Get the Class’s
__mro__
as ClassParents - For each ClassParent in ClassParents
- If the Attribute is in the ClassParent’s
__dict__
- If is a data descriptor
- Return the result from calling the data descriptor’s special method
__get__()
- Return the result from calling the data descriptor’s special method
- Break the for each (do not continue searching the same Attribute any further)
- If the Attribute is in the ClassParent’s
- If the Attribute is in Instance’s
__dict__
- Return the value as it is (even if the value is a data descriptor)
- For each ClassParent in ClassParents
- If the Attribute is in the ClassParent’s
__dict__
- If is a non-data descriptor
- Return the result from calling the non-data descriptor’s special method
__get__()
- Return the result from calling the non-data descriptor’s special method
- If it is NOT a descriptor
- Return the value
- If the Attribute is in the ClassParent’s
- If Class has the special method
__getattr__
- Return the result from calling the Class’s special method
__getattr__
.
- Return the result from calling the Class’s special method
- Raises an
AttributeError
Things to remember (from the manual)
- descriptors are invoked by the getattribute() method
- overriding getattribute() prevents automatic descriptor calls
- getattribute() is only available with new style classes and objects
- object.getattribute() and type.getattribute() make different calls to get().
- data descriptors always override instance dictionaries.
- non-data descriptors may be overridden by instance dictionaries.
Class attribute look up
Given a MetaClass “M” and a Class “C” instance of the Metaclass “M”, calling “C.name” means looking up an Attribute “name” on the Class “C” like this:
- Get the Metaclass from Class
- Call the Metaclass’s special method
__getattribute__
Inside __getattribute__
- Get the Metaclass’s
__mro__
as MetaParents - For each MetaParent in MetaParents
- If the Attribute is in the MetaParent’s
__dict__
- If is a data descriptor
- Return the result from calling the data descriptor’s special method
__get__()
- Return the result from calling the data descriptor’s special method
- Break the for each
- If the Attribute is in the MetaParent’s
- Get the Class’s
__mro__
as ClassParents - For each ClassParent in ClassParents
- If the Attribute is in the ClassParent’s
__dict__
- If is a (data or non-data) descriptor
- Return the result from calling the descriptor’s special method
__get__()
- Return the result from calling the descriptor’s special method
- Else
- Return the value
- If the Attribute is in the ClassParent’s
- For each MetaParent in MetaParents
- If the Attribute is in the MetaParent’s
__dict__
- If is a non-data descriptor
- Return the result from calling the non-data descriptor’s special method
__get__()
- Return the result from calling the non-data descriptor’s special method
- If it is NOT a descriptor
- Return the value
- If the Attribute is in the MetaParent’s
- If MetaClass has the special method
__getattr__
- Return the result from calling the MetaClass’s special method
__getattr__
.
- Return the result from calling the MetaClass’s special method
- Raises an
AttributeError
A guide to using Google’s API Client Library for JavaScript with Sheets API
Google allow users to interact with Google Apps (like Calendar, Contacts, Drive) though APIs.
Working with the API Client Library for JavaScript in a browser environment presents it’s challenges as described below.
Setup
I’m making a single page application (one html file) to test the Javascript library capabilities for using it with Google Sheets Api.
Wish to demo the following:
* create a new spreadsheet
* search for the newly created file by name
* read rows and columns
* insert new rows
Working with private data requires setting up a project at https://console.developers.google.com/project
Read the chapter “Get access keys for your application” from https://developers.google.com/api-client-library/javascript/start/start-js
Make sure to add “Drive API” here https://console.developers.google.com/project/PROJECT_KEY/apiui/apis/enabled
After “Create new client id” steps, write down the “Client ID” and make sure that JavaScript origins matches the demo server (in my case it was http://localhost:8888)
I’ll start building the demo with the help of these resources:
https://developers.google.com/api-client-library/javascript/samples/samples
https://github.com/google/google-api-javascript-client/blob/master/samples/authSample.html
The demo
This depends on “client.js” which is a single line js file like this: window.clientId = ‘aaa-bbbcccddd.apps.googleusercontent.com';
Here’s the file on GITHUB: https://github.com/florentin/google-sheets-api
Please read the code comments, they are important.
<!DOCTYPE html> <html> <head> <meta charset='utf-8' /> </head> <body> <h1>Hello Google Sheets API</h1> <em>Open the console and watch for errors and debug messages.</em> <div id="step1"> <h2>Step 1: Authorize this app with your Google account</h2> <span id="authorize-status"></span> <button id="authorize-button" style="visibility: hidden">Authorize</button> </div> <div id="step2"> <h2>Step 2: Create a Spreadsheet document in Google Drive</h2> <span id="create-status"></span> <button id="create-button">Create file in Drive</button> <em>Add a file named "blocky" to Google Drive</em> </div> <div id="step3"> <h2>Step 3: Search spreadsheet files by the name "blocky"</h2> <span id="list-status"></span> <button id="list-button">Search files (json)</button> </div> <div id="step4"> <h2>Step 4: Find the first file named "blocky" and retrieve the worksheets</h2> <span id="worksheets-status"></span> <button id="worksheets-button">Retrieve worksheets</button> </div> <div id="step5"> <h2>Step 5: Add rows to the first worksheet from the spreadsheet "blocky"</h2> <span id="rows-status"></span> <button id="rows-button">Add rows</button> </div> <script src="client.js"></script> <script type="text/javascript"> var scopes = 'https://spreadsheets.google.com/feeds', headers = { accept_json: {'Accept': 'application/json'}, accept_xml: {'Accept': 'application/atom+xml'}, send_json: {'Content-Type': 'application/json'}, send_xml: {'Content-Type': 'application/atom+xml'} }, authorizeButton = document.getElementById('authorize-button'), authorizeStatus = document.getElementById('authorize-status'), createButton = document.getElementById('create-button'), listButton = document.getElementById('list-button'), worksheetsButton = document.getElementById('worksheets-button'), rowsButton = document.getElementById('rows-button'); function handleClientLoad() { checkAuth(); createButton.onclick = handleCreateClick; listButton.onclick = handleListClick; worksheetsButton.onclick = handleWorksheetsClick; rowsButton.onclick = handleRowClick; } function checkAuth() { gapi.auth.authorize({client_id: clientId, scope: scopes, immediate: true}, handleAuthResult); } function handleAuthResult(authResult) { if (authResult && !authResult.error) { // this app is authorised authorizeButton.style.visibility = 'hidden'; authorizeStatus.innerHTML = 'Is Authorised'; //makeApiCall(); } else { authorizeButton.style.visibility = ''; authorizeButton.onclick = handleAuthClick; authorizeStatus.innerHTML = 'Not Authorised'; } } function handleAuthClick(event) { gapi.auth.authorize({client_id: clientId, scope: scopes, immediate: false}, handleAuthResult); return false; } function request(method, url, post_data, sucess_call, headers) { var oauthToken = gapi.auth.getToken(), xhr = new XMLHttpRequest(); console.info('request url', url); xhr.open(method, url); xhr.setRequestHeader('Authorization', 'Bearer ' + oauthToken.access_token); //xhr.setRequestHeader("GData-Version", "3.0"); if(headers !== undefined) { for(var key in headers) { xhr.setRequestHeader(key, headers[key]); } } xhr.onload = function() { console.info('response xhr', xhr); if(typeof sucess_call === "function") sucess_call(xhr); }; xhr.onerror = function(err) { console.error('Get Woops', err); }; xhr.send(post_data); } function get_request(url, success_call, headers) { return request('GET', url, null, success_call, headers); } function post_request(url, data, success_call, headers) { return request('POST', url, data, success_call, headers); } function json_url(url) { return url+'?alt=json'; } function serialize(obj) { return '?'+Object.keys(obj).reduce(function(a,k){a.push(k+'='+encodeURIComponent(obj[k]));return a},[]).join('&') } function xhr_json_response(xhr) { return JSON.parse(xhr.responseText); } function handleCreateClick(event) { /* To upload a file, check the Javascript example from https://developers.google.com/drive/v2/reference/files/insert */ var body = { 'mimeType': 'application/vnd.google-apps.spreadsheet', 'title': 'blocky'}, url = 'https://www.googleapis.com/drive/v2/files'; post_request(url, JSON.stringify(body), null, send_json_header); } function handleListClick(event, callback) { /* find the Drive Api reference here https://developers.google.com/drive/v2/reference/ */ var url = 'https://www.googleapis.com/drive/v2/files'+serialize({'q': 'title="blocky" and trashed=false'}); get_request(url, function(xhr) { var obj = xhr_json_response(xhr); console.info('documents', obj, obj.items); if(typeof callback === "function") { callback(obj); } }); } function handleWorksheetsClick(events) { /* Google Sheets Api has no "reference" page and i couldn't figure out a way to find a spreadsheet based on it's id. https://developers.google.com/google-apps/spreadsheets/index Make sure that the "blocky" spreadsheet has the following rows: key content url a1 a2 a3 b1 b2 b3 */ handleListClick(null, function(obj) { // This is the endpoint for getting the worksheets of a spreadsheet identified by "id" var url = 'https://spreadsheets.google.com/feeds/worksheets/'+encodeURIComponent(obj.items[0].id)+'/private/full'; /* === TROUBLE AHEAD === Google Sheets Api by default returns and accepts "application/atom+xml" content type - that's XML strings */ // Let's get an "ATOM+XML" response of the worksheets for our "blocky" spreadsheet get_request(url, function(xhr) { console.info('Request atom+xml', xhr); }, headers.accept_xml); /* The request doesn't get thought because of this error: No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8888' is therefore not allowed access. If you check the Network Headers in Chrome Developer tools, Chrome first does a "OPTIONS" request to the "url" and gets: access-control-allow-headers:authorization access-control-allow-methods:GET,OPTIONS access-control-allow-origin:http://localhost:8888 All seems fine until it makes the actual "GET" request where the "access-control-allow-origin" header dissapears and Google Chrome denies the cross origin request. */ /* Let's alter the "url" to request a "JSON" feed as described here: https://developers.google.com/gdata/docs/json */ get_request(url+'?alt=json', function(xhr) { console.info('Request json', xhr); var obj = xhr_json_response(xhr); console.log('json', obj, obj.feed.entry[0]); /* Using a JSON request (?alt=json) allows getting the data out of the worksheets. Manually add some data to the "blocky" spreadsheet in Google Drive app to see results */ var url = obj.feed.entry[0].link[0].href; get_request(url+'?alt=json', function(xhr) { var obj = xhr_json_response(xhr); console.info('Spreadsheet data', obj, obj.feed.entry); }); }, headers.accept_json); }); } function handleRowClick(events) { handleListClick(null, function(obj) { /* Adding new rows as described in the chapter "Add a list row" from https://developers.google.com/google-apps/spreadsheets/data is not working */ var url = 'https://spreadsheets.google.com/feeds/worksheets/'+encodeURIComponent(obj.items[0].id)+'/private/full'; get_request(url+'?alt=json', function(xhr) { console.info('Request json', xhr); var obj = xhr_json_response(xhr); //console.log('worksheet', obj) var url = obj.feed.entry[0].link[0].href; var post_data = '<entry xmlns="http://www.w3.org/2005/Atom" xmlns:gsx="http://schemas.google.com/spreadsheets/2006/extended"><gsx:key>x1</gsx:key><gsx:content>x2</gsx:content><gsx:url>x3</gsx:url></entry>'; post_request(url, post_data, function(xhr) { console.log('post data', xhr); /* No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8888' is therefore not allowed access. */ }, headers.send_xml); }); }); } </script> <script src="https://apis.google.com/js/auth.js?onload=handleClientLoad"></script> </body> </html>
Conclusions
- Google Sheets API is using the old “gdata API”. Read more https://developers.google.com/gdata/docs/directory and here https://developers.google.com/gdata/docs/json
- Reading operations seem to work fine with CORS (https://developers.google.com/api-client-library/javascript/features/cors) requests as long as you request JSON feeds (add ?alt=json to the requested url)
- I wasn’t able to “POST” data using CORS requests, using either “atom+xml” or “json” content types.
- Sending “POST” requests from “curl” or Python worked fine.
- All these basically mean that you can’t build Single-Page Apps for changing data in Google services (unless you use a back-end request proxy)
Blocky – the Chrome Extension that enables tagging blocks of html
This small Chrome Extension i’ve created is good for tagging divs or other blocks of html.
Let’s say you see a web page listing the latest 50 news. Using blocky extension you would be able to tag the news you like with “green”, the ones you don’t like with “red” and the ones you didn’t read with “yellow”. On the following visit you’d see the marked items.
Please check the extension here:
A better diigo bookmarklet (diigolet)
What is the ‘diigolet’
The diigolet is a Diigo.com tool used to manage a user’s bookmarks. It is a bookmarklet, a bookmark stored in a browser that contains Javascript.
The original diigolet is here:
https://www.diigo.com/tools/diigolet
How is BetterDiigolet different ?
BetterDiigolet by default opens the “bookmark” menu from the diigolet toolbar, it fills in the description and clicks all recommended tags. All is left for the user is to click “Save Bookmark”.
If it’s used on a already bookmarked url, it will color the tags previously used in ‘green’. The new recommended tags will remain ‘blue’.
This is an easy way for the user to find out if the url has been bookmarked before.
By using better_diigolet.js one can save time with clicking though all the tags and filling in the bookmark’s description field.
How it’s built
Diigo’s diigolet is loading a js file named “diigolet_b_h_b.js” which in turns loads “diigolet.js”. diigolet.js contains jQuery v1.4.3.
BetterDiigolet loads everything in better_diigolet.js, jQuery has been replaced with v1.8.3
The downside is the missing “Loading Diigolet…” tooltip.
How to install it in Firefox
- Make sure the “Bookmarks Toolbar” is visible. If it is not, go to menu View > Toolbars.
- Drag this button: BetterDiigolet up to your Bookmarks Toolbar.
Django reusable apps
Warning: this article is still a draft
Explaining the Django reusable apps in an agile way
- as a developer
- i want to easily plug the app into the project so that i can improve the productivity
- i want to customize and extend existing Django apps so that they fit the requirements
- some of the things that need customization:
- models
- e.g. make the app work with a different model class than the ones provided by the app
- e.g. create a custom model based on the one supplied by the app and add extra fields or modify the existing field definition
- forms
- e.g. the app should use a custom form
- e.g. a custom form which adds extra fields or change the definition of the existing ones
- template names
- success or redirect urls
- template variables (extra_context)
- models
1 James Bennett’s approach – the custom objects are passed as arguments to the views
- let the app’s views receive a number of parameters for which the view defined some defaults
- Django’s class based views support this strategy by default
- more on the subject:
- example
(APP/views.py)
def register(request, success_url=None,
form_class=RegistrationForm
template_name=’registration/registration_form.html’,
extra_context=None):
2 Use the settings to define the custom objects
- in settings.py, define the values for models/forms/etc which will be later used by the app
- it’s useful to prefix all the names required by a specific app, i.e. BLOGAPP_MODEL, BLOGAPP_FORM etc
- example
(settings.py)
MY_APP_MODEL = MyAppModel
(APP/views.py)
from django.conf import settings
from .models import AppModel
def register(request):
What the current approaches don’t offer
- an easy way to modify or extend the urls. The urlpatterns is just a list, it’s difficult to delete/edit/insert url definitions.
- models placed in models.py are used by Django in different db operations (i.e. syncdb).
One might want to ignore those models and let Django use customized/child Models.
- the Model and Form fields are not easy to alter
Ideas for a new solution
- make everything a class. Forms and Models are classes but the urls (or maybe the views) are not.
- move the Model definitions from models.py and allow the developer to use it’s own Models (i.e. though getters or factories)
- instead one can play with abstract models but without any concrete Models the app won’t work out of the box
- define the Form and Model fields inside a method so that the derived classes can modify the definitions
Resources
- ways to organize your app for reusability https://github.com/Natim/django-reusable-apps
- http://www.philipotoole.com/reusable-django-applications
- Django class based views https://docs.djangoproject.com/en/dev/topics/class-based-views/
- “eight spaces” – class based everything http://reinout.vanrees.org/weblog/2011/06/07/eight-spaces.html
Django project skeleton
If you need a Django project skeleton to base your work upon, please check this one:
https://github.com/florentin/django-project-skeleton
And now a Django app skeleton as well
Django settings
I’m going to talk here about various techniques regarding the Django’s settings.
Let’s assume the following project layout:
.
├── __init__.py
├── manage.py
├── settings
└── urls.py
How do I use custom settings or overwrite existing ones on my development machine ?
Answer 1
Create your custom settings file under the same parent as the existing settings.py. Let’s name the new file settings_local.py
At the bottom of settings.py, add the following:
try: from settings_local import * except ImportError: pass
Pros
- no need to change the manage.py or the wsgi file in order to apply the new settings
Cons
- hard/impossible to use different settings for different environment
Answer 2 (preferred)
Create a new directory called “settings” and move the existing setting file there. Then make a different file for each type of environment.
.
├── __init__.py
├── manage.py
├── settings
│ ├── __init__.py
│ ├── development.py
│ ├── production.py
│ └── settings.py
└── urls.py
settings.py will include the default Django settings, probably the file created by django-admin.py
development.py will hold settings/overwrites needed for the development environment.
The extra settings files (production.py, development.py, etc) will extend the existing settings.py (or another parent file which in turn extends settings.py) and add their own settings customizations.
This could be your development.py file:
from .production import * DEBUG = True TEMPLATE_DEBUG = True DJANGO_SERVE_PUBLIC = True PREPEND_WWW = False SEND_BROKEN_LINK_EMAILS = False # APP: debug_toolbar MIDDLEWARE_CLASSES += ( "debug_toolbar.middleware.DebugToolbarMiddleware", ) INSTALLED_APPS += ( "debug_toolbar", ) DEBUG_TOOLBAR_CONFIG = { 'INTERCEPT_REDIRECTS': False, } TEMPLATE_CONTEXT_PROCESSORS += [ 'django.core.context_processors.debug' ] Where production.py is:
from .settings import * import os.path PROJECT_ROOT = os.path.join(os.path.abspath(os.path.dirname(__file__)), '../') DEBUG = False TEMPLATE_DEBUG = False
Once your settings setup is in place, all you have to do is change manage.py and your WSGI file.
The manage.py file could now look like this:
#!/usr/bin/env python from django.core.management import execute_manager import sys, os PROJECT_ROOT = os.path.abspath(os.path.dirname(__file__)) sys.path.insert(0, PROJECT_ROOT) try: import settings.development # Assumed to be in the same directory. except ImportError, e: import sys sys.stderr.write("Error: Can't find the file 'settings.py' in the directory containing %r. It appears you've customized things. You'll have to run django-admin.py, passing it your settings module. (If the file settings.py does indeed exist, it's causing an ImportError somehow.) " % __file__) sys.exit(1) if __name__ == "__main__": execute_manager(settings.development)
In the same time, your WSGI file would use settings.production:
os.environ[“DJANGO_SETTINGS_MODULE”] = “settings.production”
Pros
- easy to create settings for each of your environments (production, development, etc)
- it’s a great way to keep your settings organized, easy to find and edit.
- easier to reuse your settings for other projects. For example, we could use the same development.py file (as shown above) with other production.py settings.
Cons
- you have to change the manage.py and the WSGI file, which might not be possible on your client server.
Other Django settings tips
- do not use your project name in the settings, i.e. use ROOT_URLCONF = ‘urls’ and not ROOT_URLCONF = ‘myproject.urls’, use INSTALLED_APPS = (“appname”,) and not INSTALLED_APPS = (“myproject.appname”, ). You will then be able to easily move settings and applications between one project to another
- use calculated paths for TEMPLATE_DIR, MEDIA_ROOT, STATIC_ROOT etc, i.e. PROJECT_ROOT = os.path.abspath(os.path.dirname(__file__)) and then MEDIA_ROOT = os.path.join(PROJECT_ROOT, “media”)
This means business
This is how it goes …
Father: Son, I would like to be the one who chooses your future wife.
Son: No way
Tata: That girl is Bill Gates’s doughter
Son: AAA, then it is ok.
Father goes to Bill Gates
Father: I want my son to marry your daughter.
Bill Gates: No chance
Dad: My son is CEO of the World Bank
Bill Gates: aa, then it is ok.,
Father goes to World Bank President
Tata: I would like to offer my son as CEO of the World Bank
Presenintele: No chance
Dad: My son is the future husband of Bill Gates’s doughter
President: Oh, that’s ok.
This means BUSINESS