Notes On Django
Abstract
Notes on Django.
Go through initial Django polls application tutorial:
https://docs.djangoproject.com/en/1.7/intro/tutorial01/
Use virtual env -- easy to manage different python/django versions:
$ virtualenv -p /usr/bin/python2.7 /opt/penv/p2
$ . /opt/penv/p2/bin/activate
Install django within virtual env :
$ pip freeze # output is empty -- no additional modules.
$ pip install Django==1.6.10 # Latest GA is Django==1.7.5
$ pip install mysqlclient==1.3.6
$ pip install djangorestframework==3.1.0
$ pip install django-rest-swagger # Used for auto generation of /docs
$ pip freeze # installed django modules is displayed.
Create new django project:
$ which django-admin.py
/opt/penv/p2/bin/django-admin.py
$ django-admin.py startproject mysite
See Also: django-admin.py help
django-admin.py help --commands
django-admin.py help <command>
(p2)/opt/python/django/tutorial/mysite $ls
db.sqlite3 manage.py mysite
$ ls mysite
settings.py urls.py wsgi.py ...
The manage.py is a thin wrapper on top of django-admin.py -- makes it easy
to run django.
The inner mysite directory is the actual Python package for your project.
Its name is the Python package name you’ll need to use to import anything inside it
(e.g. mysite.urls).
mysite/__init__.py: An empty file to mark this as Python package.
mysite/settings.py: Settings/configuration for this Django project.
mysite/urls.py: The URL declarations for this Django project;
mysite/wsgi.py: An entry-point for WSGI-compatible web servers (e.g. apache)
to serve your project.
Note: There is no admin.py in mysite since urls.py contains admin url registration logic.
Start server: $ python manage.py runserver [8000]
Create application: $ python manage.py startapp myapp :
myapp/
__init__.py
admin.py
models.py
tests.py
views.py
urls.py
Note: Application is created at top level along side project as separate pkg.
Configure your mysql database to be used by editing mysite/settings.py as follows:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'mydb',
'HOST': '127.0.0.1',
'PORT': 3306,
'USER': '',
'PASSWORD': ''
}
}
Then do the following:
$ manage.py syncdb (version django 1.6)
For version django 1.7:
$ manage.py makemigrations <appname>
manage.py migrate <appname>
(e.g.) ./manage.py migrate myapp 0001
$ manage.py syncdb
The above command will create initial tables and also superuser admin account. You can use the username: admin with any password (say admin)
Explore following urls:
http://localhost:8000/
http://localhost:8000/admin [ Admin interface. Create additional entries ]
http://localhost:8000/myapp
http://localhost:8000/docs
The directory structure is not rigid. The primary settings file mysite.settings is hard coded in manage.py:
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "mysite.settings")
To generate models.py from existing SQL tables, use :
./manage.py inspectdb
The entry point for application is specified here in mysite.settings:
WSGI_APPLICATION = 'mysite.wsgi.application'
runserver/runfcgi invokes built-in development server by locating the WSGI application using above setting.
To use WSGI appserver like gunicorn or apache (with mod_wsgi):
WSGIScriptAlias / /path/to/mysite.com/mysite/wsgi.py (httpd.conf)
WSGIPythonPath /path/to/mysite.com:/my/virtualenv/python/site-packages:... (httpd.conf)
(Use python -i ; import sys; print sys.path ; )
To serve static files, you must use another webserver like lighttpd or apache. The runserver dev server does not serve them.
However devserver serves the default static admin files. (only admin files not other static). The admin files live in (django/contrib/admin/static/admin) of the Django distribution. For apache, you have to take some steps to include these admin files path. (See django-admin.py collectstatic command)
Write custom django scripts: http://eureka.ykyuen.info/2014/08/26/django-create-custom-django-admin-command/
From model classes, you can get all instances using MyModel.objects.all() which returns QuerySet object:
City.objects.all() # Returns QuerySet which is iterable & contains SQL.
City.objects.all().values() # Gives list of dict objects.
Always use an APIView class or @api_view function for views that return Response objects. This performs content negotiation and select the appropriate renderer for the response. Using this we can make default response is JSON (using wget), but browser displays browsable API.
Django follows MVC pattern - or some call MTV (Model-Template-View) pattern. This view is more like Controller.
City.objects is the default Manager object for the model. You can override this manager or write your own additional manager. See http://www.effectivedjango.com/orm.html#writing-models
Use factory boy for model factory generator which creates default complex instance for class (useful for testing, etc). See https://pypi.python.org/pypi/factory_boy
Use related name for one-to-many relation: See http://stackoverflow.com/questions/2642613/what-is-related-name-used-for-in-django
Foreign keys are not traversed unless you use select_related: Contact.objects. select_related('address'). filter(last_name = 'Yergler') Without using select_related, each time you access c.address, a fresh query is executed, may be inefficient.
Can use raw SQL: Contact.objects.raw('SELECT * FROM contacts WHERE last_name = %s', [lname])
Q objects can be used for OR queries. Python object overloading is supported by writing functions for e.g. __lt__() :
Contact.objects.filter( Q(state='OH') | Q(email__endswith='osu.edu'))
Poll.objects.get( Q(question__startswith='Who'), Q(pub_date=date(2005, 5, 2)) | Q(pub_date=date(2005, 5, 6))
The comma acts like AND operator.)
You can do outerjoin by doing backwards relation.
- See https://docs.djangoproject.com/en/dev/topics/db/queries/#lookups-that-span-relationships :
>>> qs = Department.objects.filter(departmentvolunteer__isnull=True).values_list('name', flat=True)
>>> print(qs.query)
SELECT "app_department"."name" FROM "app_department" LEFT OUTER JOIN
"app_departmentvolunteer" ON ( "app_department"."id" = "app_departmentvolunteer"."department_id" )
WHERE "app_epartmentvolunteer"."id" IS NULL
You can perform raw SQL that returns models. Or can use it to return arbitrary object. See https://docs.djangoproject.com/en/1.7/topics/db/sql/ :
>>> for p in Person.objects.raw('SELECT * FROM myapp_person'): // Result is of type Person
... print(p)
>>> objects = Manager.raw(raw_query, params=None, translations=None)
// For ultimate control, use raw cursors:
from django.db import connection
def my_custom_sql(self):
cursor = connection.cursor()
cursor.execute("UPDATE bar SET foo = 1 WHERE baz = %s", [self.baz])
cursor.execute("SELECT foo FROM bar WHERE baz = %s", [self.baz])
row = cursor.fetchone()
return row
Nice Overview: See http://blog.kevinastone.com/getting-started-with-django-rest-framework-and-angularjs.html
Django by example: http://lightbird.net/dbe2/
Perfect to conduct django workshops: http://djangogirls.org/resources/
Use django-extensions which adds nice manage.py commands.
Json serialization is different from Pickle object serialization.
Use higherlevel API: See http://www.dabapps.com/blog/higher-level-query-api-django-orm/
In pure REST based API, you can remove django.contrib.admin, django.contrib.sessions, and django.contrib.messages from the INSTALLED_APPS. The new INSTALLED_APPS include rest_framework and rest_framework.authtoken.
HTTP Basic Auth: Server says: WWW-Authenticate: Basic realm="nmrs_m7VKmomQ2YM3:" Client says: Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ== where username:password is base64 encoded.
Use http://django-rest-auth.readthedocs.org/en/latest/api_endpoints.html to create/manage new users. How to add custom fields ??
Use UserProfile to store additional info: See http://buildthis.com/customizing-djangos-default-user-model/
Custom permissions can be programmatically added: https://docs.djangoproject.com/en/1.6/topics/auth/default/#topic-authorization
Custom permissions for given model, can be specified in class Meta: https://docs.djangoproject.com/en/1.6/topics/auth/customizing/#custom-permissions
django.contrib.auth.User is the default settings.AUTH_USER_MODEL ; However different user model may be configured.. so you should use: django.contrib.auth.get_user_model() to get User model. Code for using custom user is here: https://docs.djangoproject.com/en/1.6/topics/auth/customizing/
For using API class based views: http://www.django-rest-framework.org/tutorial/3-class-based-views/ For function based API http://www.django-rest-framework.org/tutorial/2-requests-and-responses/
What we want:
There is serializer in django core that can serialize into xml, json, etc :
....
http://timsaylor.com/index.php/2012/05/21/convert-django-model-instances-to-dictionaries/
The django rest framework supports working with serializers :
pip install djangorestframework
pip install pygments # code highlights
INSTALLED_APPS = ( ... 'rest_framework', )
from rest_framework import serializers
from snippets.models import Snippet # Snippet is our model class.
Approach 1 - Low level serializer :
class SnippetSerializer(serializers.Serializer):
pk = serializers.IntegerField(read_only=True)
title = serializers.CharField(required=False, allow_blank=True, max_length=100)
code = serializers.CharField(style={'base_template': 'textarea.html'})
linenos = serializers.BooleanField(required=False)
language = serializers.ChoiceField(choices=LANGUAGE_CHOICES, default='python')
style = serializers.ChoiceField(choices=STYLE_CHOICES, default='friendly')
def create(self, validated_data):
"""
Create and return a new `Snippet` instance, given the validated data.
"""
return Snippet.objects.create(**validated_data)
def update(self, instance, validated_data):
"""
Update and return an existing `Snippet` instance, given the validated data.
"""
instance.title = validated_data.get('title', instance.title)
instance.code = validated_data.get('code', instance.code)
instance.linenos = validated_data.get('linenos', instance.linenos)
instance.language = validated_data.get('language', instance.language)
instance.style = validated_data.get('style', instance.style)
instance.save()
return instance
In your live server environment, you’ll need to tell your WSGI application what
settings file to use. Do that with os.environ:
import os
os.environ['DJANGO_SETTINGS_MODULE'] = 'airsite.settings'
Consider urls.py :
from django.conf.urls import patterns, include, url
from django.contrib import admin
from rest_framework import routers, serializers, viewsets
admin.autodiscover()
def welcome(request):
return HttpResponse('Welcome to Booking Engine System!. Use One of /apiadmin or /admin!')
urlpatterns = patterns('', # This is string prefix. e.g. 'myapp.views' or 'views' etc.
url(r'^admin/', include(admin.site.urls)),
url(r'^apiadmin/', include('adm.urls', namespace='adm')),
# Following is optional -- used only for browsable API page to have login/logout links.
url(r'^api-auth/', include('rest_framework.urls', namespace='rest_framework')),
url(r'^', welcome),
)
In simplest form, url(regex, func), invokes function on regex url match.
Signature:
patterns is a module function which returns a list. e.g []
patterns(prefix (e.g. 'myapp.views'), url-config1, url-config2, ...)
which returns List of RegexUrlPattern or RegexURLResolver ;
RegexURLResolver(regex, urlconf_module, kwargs, app_name=app_name, namespace=namespace)
RegexURLPattern(regex, callback, default_args=None, name=None):
url() returns RegexUrlResolver or RegexURLPattern;
patterns() ultimately reduces them to list of RegexURLPattern.
url(regex, view, kwargs=None, name=None, prefix='')
where view = (urlconf_module, app_name, namespace) | func | include('appname.urls') | urlconf_module (e.g. app.urls)
include('appname.urls') converts the string to imported module.
ViewClassName.as_view() just returns a dispatch function which invokes get() etc class methods.
restframework View class should have permission_classes defined (must) and have get(), post(), etc methods defined :
class MyView(APIView):
permission_classes = (IsAuthenticated,)
@staticmethod
def post(self, request, format=None):
Why would you use ModelAdmin class ? Ans: To reorder fields or create fieldsets or limit the fields or to create related objects (whose foriegnkey points to this) :
class QuestionAdmin(admin.ModelAdmin):
fields = ['pub_date', 'question_text']
# OR ...
fieldsets = [
(None, {'fields': ['question_text']}),
('Date information', {'fields': ['pub_date']}),
]
# inlines = [ChoiceInline]
admin.site.register(Question, QuestionAdmin)
How do you receive component URL parameters ? e.g. /polls/32/ ? :
url(r'^(?P<question_id>\d+)/$', views.detail, name='detail'), ...
def detail(request, question_id):
....
How do you pass the model to template and how do you render ? :
def index(request):
latest_question_list = Question.objects.order_by('-pub_date')[:5]
context = {'latest_question_list': latest_question_list}
return render(request, 'polls/index.html', context)
# polls/templates/polls/index.html
{% if latest_question_list %}
<ul>
{% for question in latest_question_list %}
<li><a href="/polls/{{ question.id }}/">{{ question.question_text }}</a></li>
{% endfor %}
</ul>
{% else %}
<p>No polls are available.</p>
{% endif %}
Use pycscope -i filelist.txt and :cs add /tmp/cscope.out from vim; cs find f for file or module. g for definition.
What is django generic views used for ? :
- To view given model details using template (generic.DetailView)
- To view list of objects returned by Queryset using template (generic.ListView)
- To render given template. (TemplateView.as_view(template_name="about.html"))
- Provides ability to customize the view using keyword args:
class MyView(django.views.generic.View):
maxcount = 1000
MyView.as_view(maxcount=500)
- Tie given model to the view so that functions like self.get_object() works using
pk value provided in URL conf keyword. e.g.: url(r'^author/(?P<pk>\d+)/interest/$', ...)
-
Where is my raw JSON body string that has been Posted ? It is available as string using request.body of handler. The request.data is the Python object.
What is the use of viewset in restframework ?
A ViewSet class is simply a type of class-based View, that does not provide any method handlers such as .get() or .post(), and instead provides actions such as .list() and .create().
Viewset has associated queryset or model and serializer. The list(), retrieve() methods returns JSON response for the object instance(s).
We always use viewsets with routers though we could use this directly with myviewset.as_view() :
router = DefaultRouter()
router.register(r'users', UserViewSet)
urlpatterns = router.urls
The main advantage is that it generates url patterns with primary key 'users/pk' for lookup and use appropriate method for post, put, get, etc.
ModelViewSet is a huge benefit to directly manipulate with model object.
A viewset that provides default create(), retrieve(), update(), partial_update(), destroy() and list() actions.
In theory, you can use GenericViewSet and override, update(), etc functions. But you will hardly use it that way. Instead, you may use plain view. It is useful only along with model.
It is useful to write specific actions on model instances like below:
For urls ^users/{pk}/set_password/$ and ^users/{pk}/unset_password/$
You can write set_password() function and annotate with detail_route(methods=['post']).
See https://github.com/rtfd/readthedocs.org/blob/master/readthedocs/comments/views.py for good example of viewset.
What are some builtin Django db objects manipulation functions ?
How to handle query parameters ?
Good example from https://github.com/rtfd/readthedocs.org/blob/master/readthedocs/comments/views.py :
class CommentViewSet(ModelViewSet):
serializer_class = DocumentCommentSerializer
permission_classes = [CommentModeratorOrReadOnly, permissions.IsAuthenticatedOrReadOnly]
def get_queryset(self):
qp = self.request.QUERY_PARAMS
if qp.get('node'):
try:
node = DocumentNode.objects.from_hash(version_slug=qp['version'], page=qp['document_page'], ...)
queryset = DocumentComment.objects.filter(node=node)
except KeyError:
raise ParseError('To get comments by node, you must also provide page, version, and project.')
except DocumentNode.DoesNotExist:
queryset = DocumentComment.objects.none()
elif qp.get('project'):
queryset = DocumentComment.objects.filter(node__project__slug=qp['project'])
else:
queryset = DocumentComment.objects.all()
return queryset
Migrations are the ability to handle auto schema update, when you change the model. These are handled differently between Django 1.6 and 1.7 ;
South vs Django migrations :
South Migration:
python manage.py syncdb
python manage.py schemamigration <appname> --initial
./manage.py migrate <appname>
After any updates to models :
python manage.py schemamigration <appname> --auto
./manage.py migrate <appname>
Django 1.7 Migration:
python manage.py makemigrations <appname>
./manage.py migrate <appname>
e.g. python manage.py migrate polls 0001
After any updates to models just run above 2 commands again.
If you have SQL, but you want to create models from them, use:
$ python manage.py inspectdb > models.py
In essense:
- Change your models (in models.py).
- Run python manage.py makemigrations appname to create migrations for those changes
- Run python manage.py migrate to apply those changes to the database.
See Also:
https://docs.djangoproject.com/en/1.7/topics/migrations/
https://realpython.com/blog/python/django-migrations-a-primer/
https://docs.djangoproject.com/en/1.7/howto/initial-data/
Testing unmanaged models application: See http://www.caktusgroup.com/blog/2010/09/24/simplifying-the-testing-of-unmanaged-database-models-in-django/ Slight modification of above: http://shanedowling.com/unit-testing-unmanaged-django-models/
class Model1(models.Model):
id = models.CharField(max_length=200, primary_key=True)
name = models.CharField(max_length=200, blank=True, null=True)
class Meta:
managed = False
db_table = 'table_1_name'
class Model2(models.Model):
# model1_id = models.CharField(max_length=200, blank=True, null=True)
model1 = models.ForeignKey(Model1)
# now model1 is available like object: self.model1.name is now available!!!
class Meta:
managed = False
db_table = 'table_2_name'
OneToMany relational objects can be made available easily by using foreignkey.
See https://docs.djangoproject.com/en/1.6/ref/models/relations/ to find how to access related objects.
django-debug-toolbar :
$ pip install django-debug-toolbar # settings.py
INSTALLED_APPS = ( debug_toolbar, )
MIDDLEWARE_CLASSES = ( debug_toolbar.middleware.DebugToolbarMiddleware, )
INTERNAL_IPS = (127.0.0.1,)
save() is not thread safe. Use update()
django-sentry log viewer.
For asynchronity, use celery protocol (client library) with RabbitMQ as message broker. Other message brokers are also supported.
Use apache2-mpm-worker (not prefork) for better scaling with mod_wsgi in daemon mode. (not embedded mode)
SQLAlchemy is ORM alternative to builtin Django ORM.
pip install djangorestframework==3.1.0 Gives you web browsable API. Add this to: your INSTALLED_APPS = ( ... 'rest_framework',)
Follow http://www.django-rest-framework.org/ steps.
See http://ngenworks.com/blog/how-django-rest-framework-changed-my-life/
Use https://github.com/marcgibbons/django-rest-swagger for auto doc generation:
pip install django-rest-swagger
INSTALLED_APPS = ( ... 'rest_framework_swagger', .. ) ;
patterns = ('', ... url(r'^docs/', include('rest_framework_swagger.urls')),)
Use 'DEFAULT_AUTHENTICATION_CLASSES': ( 'rest_framework.authentication.BasicAuthentication', )
Viewsets by default support list, create, retrieve, update, partial_update, destroy methods.
Viewsets support @link and @action annotation in 1.* versions; Django 2.4+ support @detail_route and @list_route