Software Maniacs blog » Djangohttps://softwaremaniacs.org/blog/category/django/en/2019-08-21T12:07:43.638303-07:00ManiacIvan Sagalaev on programming and web developmenthttp://softwaremaniacs.org/media/sm_org/style/photo.jpgSM.Org software update 2012
2012-05-26T01:22:39.580000-07:00https://softwaremaniacs.org/blog/2012/05/23/smorg-update-2012/en/Over the course of a few recent weeks I updated this site to a more modern software and revised some previously made choices. This one was loooong overdue considering that I still ran Ubuntu 9.10 before the update meaning that the system was almost 3 years old. Here are some ...
<p>Over the course of a few recent weeks I updated this site to a more modern software and revised some previously made choices. This one was loooong overdue considering that I still ran Ubuntu 9.10 before the update meaning that the system was almost 3 years old.</p>
<p>Here are some mostly useless but probably fascinating notes about it.</p>
<p><a name=more></a></p>
<h2>Core system upgrades</h2>
<p>Running a comparatively low-load site I had a luxury not to plan for complex procedures minimizing downtime. What I did was just SSHing on the host and running <code>do-release-upgrade</code> under "screen" five times in a row. <a href="http://library.linode.com/troubleshooting">Linode's upgrade docs</a> were holding my hand during the process.</p>
<p>Currently the site runs on Ubuntu 12.04 "Precise" and 3.0 Linux kernel. It's good to have modern packages!</p>
<p>Notable moments (read: long downtimes) during the upgrades:</p>
<ul>
<li>
<p>Ubuntu's integrated mail stack was good back then and these days it's just wonderful! Upon installing a single package "mail-stack-delivery" you get yourself a completely set up mail server that can send mail, accept mail from outside and that provides an IMAP interface to your mailbox. It even does necessary SSL key generation magic for me, so I can do authorization from my mail client securely.</p>
</li>
<li>
<p>What I struggled with during all the mail upgrades was Yandex' anti-spam software that relies on manual creative editing of /etc/postfix/master.cf. I still use it simply because it <em>works</em> but I should probably look for some more actively supported solutions eventually.</p>
</li>
<li>
<p>Postgres updated to 9.x. Database conversion went without a hitch. The only problem was that now Ubuntu seems to be not as conservative in memory settings as it was before so Postgres refused to run due to low memory on my machine (756M) until I reduced the amount of "buffers" (I have no idea what it is). Seems to run pretty smooth so far :-).</p>
</li>
<li>
<p>The default Python is now 2.7. The change reminded me harshly that I was stupid enough to install some non-debianized Python packages into the system directory of which Ubuntu hadn't a faintest idea. As a result new Python didn't see them. I'm still trying to figure out how best to solve this (more on that later).</p>
</li>
<li>
<p>The longest downtime was caused by upgrading Django to the newest trunk version. I neglected it for the time long enough for many DeprecationWarnings to become exceptions. Spent some quality time refreshing my code base. One thing remaining unresolved is that now Django <em>insist</em> on <code>MEDIA_URL</code> and <code>STATIC_URL</code> to have different values. I have no idea why it has to be a hard requirement. My problem is that I don't use uploaded media and hence probably shouldn't use <code>MEDIA_URL</code> at all. Except that my forum mutants are generated on the server using an <code>ImageField</code> which does use <code>MEDIA_URL</code> for storage. Still, nothing what I can't deal with in due time, just have to figure out the best way.</p>
</li>
</ul>
<h2>New web stack</h2>
<p>I moved from the old "lighttpd + FastCGI with flup" setup to the new one with "nginx + uwsgi". For some personal, completely subjective reason which I don't even remember anymore I always preferred lighttpd over nginx. However this was the case where the best maintainer won. It seems that nginx is more actively developed and uwsgi has most mind share. I also suspect all this is old news to everyone except me :-). But what actually bought me was the built-in support for uwsgi in nginx. I love integrated solutions! It means that I will have to write less of stupid glue-code in files that I will later forget.</p>
<p>The config file for nginx turned out to be much simpler than the one for lighttpd. It looks like it was specifically designed for the kind of tasks that web server admins do rather than being not-exactly-Turing-complete Perl-like code that happen to cover most use-cases with the enough amount of regexps.</p>
<p>OK, I just have to show you an example. Here's what I needed to do to handle some legacy redirects with lighttpd (heavily stripped and simplified):</p>
<pre><code>fastcgi.server = (
"/fcgi" => (
(
"socket" => "/var/run/sm_org/fcgi.socket" ,
"check-local" => "disable",
)
)
)
url.redirect = (
"^/soft/tags/(.*)" => "/soft/tagsfield/$1",
)
url.rewrite-once = (
"^(/soft/tags/)(.*)" => "$1$2",
"^/(.*)$" => "/fcgi/$1",
)
</code></pre>
<p>I don't even mind the infamous rewrite hack to connect the FastCGI backend (got used to it). But having a redirect <em>and</em> a corresponding no-op rewrite to make the former work… Seriously? (Before you ask, there <em>is</em> a reason why those redirects are not handled by Django code.)</p>
<p>Here's the nginx version that simply looks like a thing like that should look:</p>
<pre><code>server {
server_name softwaremaniacs.org;
rewrite ^/soft/tags/(.*) /soft/tagsfield/$1 permanent;
location / {
include uwsgi_params;
uwsgi_pass unix:/var/run/uwsgi/sm_org.socket;
}
}
</code></pre>
<p>The uwsgi part of the story however wasn't that bright. What inspired my to try it out was this <a href="http://tghw.com/blog/multiple-django-and-flask-sites-with-nginx-and-uwsgi-emperor">article about running multiple sites under uwsgi "Emperor mode"</a>. But since I don't run multiple sites I decided to run it first in a <em>simple</em> way.</p>
<p>With only one site to run I ditched the idea of having a separate upstart config for the master uwsgi process and a separate config for a site. Instead I put all the parameters in the upstart script itself as arguments to the uwsgi command.</p>
<p>Then I spent some time figuring out why it simply didn't run. Turns out that uwsgi no longer supports <code>--module</code> argument contrary to the statement that the config file keys are equivalent the command line arguments. The fact that <a href="https://docs.djangoproject.com/en/dev/howto/deployment/wsgi/uwsgi/">Django uwsgi doc</a> also refers to <code>--module</code> didn't help. Neither the fact that upstart has no diagnostics whatsoever (or I couldn't find one). </p>
<p>So I reconfigured everything toward the Emperor mode.</p>
<p>Then I spent some time trying to convince nginx running under "www-data" to talk to uwsgi running under my local user. Yes, this is uncommon, but solving <em>that</em> problem was way out of scope of my intentions. Anyway, apparently there's the wonderful <code>chmod-socket</code> option in uwsgi that solved it. Also I could probably use a TCP socket (by the way, does anyone know what's the practical difference and why everyone seems to prefer using unix sockets?)</p>
<p>Then I spent some time looking helplessly at nginx complaining that it doesn't get data from the backend and uwsgi writing some logs that didn't seem to have anything to do with it. Apparently the important line in those logs was: </p>
<pre><code>-- unavailable modifier requested: 0 --
</code></pre>
<p>… which means "you don't have uwsgi-plugin-python installed". Obvious, right? :-)</p>
<p>Now, I don't exactly blame Ubuntu (or Debian?) maintainers who, after splitting uwsgi functionality into plugins, didn't think it necessary to include or recommend a single one of them to make uwsgi, you know, <em>useful</em>. Neither do I blame uwsgi maintainers for this cryptic error message. And neither do I blame myself for overlooking a warning in the <a href="http://projects.unbit.it/uwsgi/wiki/Quickstart">uwsgi Quickstart guide</a> that would solve my problem. </p>
<p>What I blame is the whole way of setting up computer software that we established over the last century. The culture of making every imaginable little thing configurable first, then forcing multitude of users to solve a few similar configuration task a million times over… But that's just frustration so please don't mind me!</p>
<p>OK, here are my uwsgi configs in case you wondered (but please use official docs whenever you can).</p>
<p>The upstart script responsible for running the uwsgi Emperor <code>/etc/init/uwsgi.conf</code>:</p>
<pre><code>start on runlevel [2345]
stop on runlevel [06]
exec /usr/bin/uwsgi \
--emperor /etc/uwsgi/apps-enabled \
--uid maniac \
--gid maniac
</code></pre>
<ul>
<li>You <em>do</em> want to run the Emperor mode even if you have only one backend simply because then you can restart it gracefully without using <code>sudo</code>. Just chmod your site config to own it yourself.</li>
<li>You'll probably want to use "www-data" for uid and gid, though I personally find it more convenient to run my code under my local user.</li>
</ul>
<p>The config describingn instance of the site backend <code>/etc/uwsgi/apps-available/sm_org.ini</code>:</p>
<pre><code>[uwsgi]
master = 1
chdir = /home/maniac/sm_org
module = wsgi
processes = 5
max-requests = 1000
plugins = python
socket = /var/run/uwsgi/sm_org.socket
chmod-socket = 777
</code></pre>
<ul>
<li>I have no idea why you need the "master" option but it seems that everyone uses it. Let's keep everyone happy!</li>
<li><code>chdir</code> into the project's directory is needed because my code does relative imports of project apps located in that directory. Yours probably does it too.</li>
<li><code>module = wsgi</code> means the <code>wsgi.py</code> in the project directory that <a href="https://docs.djangoproject.com/en/dev/howto/deployment/wsgi/">exports Django wsgi app instance</a>. In your case it might be called <code>projectname.wsgi</code> if you follow current Django project layout.</li>
<li><code>plugins = python</code> is what makes uwsgi actually run Python code. You can skip "http" that some docs suggest here if you don't use your uwsgi backend as an HTTP server.</li>
<li><code>chmod-socket = 777</code> is what allows my "www-data" owned nginx talk to the "maniac" owned uwsgi backend. You might not need it if you run them both under the same user or use a TCP socket.</li>
</ul>
<h2 id=virtualenv>virtualenv</h2>
<p>I have one unresolved question right now. OK, there's actually more than one but this one occupies me most: to use or not to use <a href="http://www.virtualenv.org">virtualenv</a>. </p>
<p>Here are my thoughts:</p>
<ul>
<li>
<p>I'm not a hosting company working for external clients so I don't run multiple sites with different set of packages. This is not my use-case for virtualenv. I'm quite happy maintaining my whole Django codebase using the trunk version of Django.</p>
</li>
<li>
<p>I cannot rely only on system Python packages simply because some of them Ubuntu doesn't provide. <a href="http://pypi.python.org/">PyPI</a> is <em>the</em> place where all new packages live now and I want to use it conveniently.</p>
</li>
<li>
<p>Installing packages with pip into the system site-packages is broken and out of the question. This is where the idea of using virtualenv comes up. But probably I could just tell pip to use a specific installation directory? I couldn't find the option for it.</p>
</li>
<li>
<p>What I don't like about virtualenv is that it makes my life harder. I should either "activate" my single environment all the time or use explicit paths everywhere to run commands. If this is how everyone does this then the world definitely gone mad :-). I'd rather keep my current way with tweaking PYTHONPATH . But then I'll still have the problem of pip trying to install everything in site-packages :-(.</p>
</li>
</ul>
<p>What a man to do?Why I don't like class-based generic views
2012-05-10T21:11:41.231000-07:00https://softwaremaniacs.org/blog/2011/10/06/class-based-generic-views/en/… because I have to replace this: def category(request, slug, language): translation.activate(language or 'ru') category = get_object_or_404(models.Category, slug=slug) return object_list(request, template_name = 'marcus/category.html', queryset = models.Article.public.language(language).filter(categories=category), paginate_by = settings.MARCUS_PAGINATE_BY, extra_context = { 'category': models.Translation(category, language), 'language': language, }, ) with this: class Category(generic.ListView): template_name = 'marcus/category.html' paginate_by = settings.MARCUS_PAGINATE_BY def ...
<p>… because I have to replace this:</p>
<pre><code>def category(request, slug, language):
translation.activate(language or 'ru')
category = get_object_or_404(models.Category, slug=slug)
return object_list(request,
template_name = 'marcus/category.html',
queryset = models.Article.public.language(language).filter(categories=category),
paginate_by = settings.MARCUS_PAGINATE_BY,
extra_context = {
'category': models.Translation(category, language),
'language': language,
},
)
</code></pre>
<p>with this:</p>
<pre><code>class Category(generic.ListView):
template_name = 'marcus/category.html'
paginate_by = settings.MARCUS_PAGINATE_BY
def get_queryset(self):
self.category = get_object_or_404(models.Category, slug=self.args[0])
return models.Article.public.language(self.args[1]).filter(categories=self.category)
def get_context_data(self, **kwargs):
translation.activate(self.args[1] or 'ru')
context = super(Category, self).get_context_data(**kwargs)
context.update({
'category': models.Translation(self.category, self.args[1]),
'language': self.args[1],
})
return context
</code></pre>
<p><a name=more></a></p>
<p>Now, these snippets might not look strikingly different from the first sight so let me break them down for you and explain my point:</p>
<ul>
<li>
<p>When your code is a function it is this function that actually works and calls other functions. When your code lives in overridden methods, something else works elsewhere and calls your code. This is called "<a href="http://en.wikipedia.org/wiki/Inversion_of_control">inversion of control</a>" and it makes code harder to read because you no longer see why and in which order things happen. This is not necessarily a bad thing (the whole framework works by inverting control, after all) but in this particular case it <em>did</em> make me think harder. In the original code I call <code>translation.activate</code> before anything else to make sure that any code that might depend on current language will have it available. Here I don't really know in which order <code>get_queryset</code> and <code>get_context_data</code> are called and the only reliable way to activate translation early is to override another method — <code>dispatch</code> — that calls both of them. But it makes code even hairier.</p>
</li>
<li>
<p>You're bound to have some boilerplate code when subclassing. Here I have two method signatures that have nothing to do with my specific domain, I just have to look them up in the docs and repeat to the letter. And I also have to call <code>super()</code>, not the prettiest thing in Python and it makes me to repeat the class name.</p>
</li>
<li>
<p>Since my code is now in two places the only way to have a variable known to both of them is to put it in the class instance — the <code>self.category</code> in this case. It doesn't really belong there, it was just a local temporary variable but after it became a class attribute it looks just as important and global as any other "real" attribute.</p>
</li>
<li>
<p>I lose descriptive names of arguments that I had in a function. Now they are available as opaque <code>self.args[..]</code> values. True, I could assign them to local variables but doing it every time is just silly…</p>
</li>
<li>
<p>And of course, the code just got obviously longer. </p>
</li>
</ul>
<p>By the way, this is not the worst refactoring that I had to deal with during my yesterday's crusade on deprecation warnings. But I decided to be merciful on you :-).</p>
<h2>Philosophy</h2>
<p>I often hear that class based generic views are better because they are more flexible and extensible. What people miss is that extensibility doesn't come <em>at all</em> from the fact that they are implemented as classes. Extensibility exists only in those places that framework explicitly defines as extensibility points. Everything that you can do by overriding a method or setting an attribute you can do with a function passing values and callables as arguments. These two paradigms are thus equivalent in terms of functionality.</p>
<p>Still, it probably makes perfectly good sense for Django itself to have generic views as classes because users demand insane level of extensibility for them. We'll never know if they would look prettier were they implemented as functions because nobody volunteered to do that. Probably readability of classes does indeed scale better along the axis of use-case complexity. </p>
<p>What I'm really trying to show is that in case of the majority of <em>user</em> code there's no point at all in having views defined as classes. Your code is never going to be as generic as Django's (unless you're building a framework) and functions just make better sense. I, for one, simply implemented a custom short version of <code>object_list</code> and everything just worked.Core devs on the future of Django
2011-10-03T23:31:09.545000-07:00https://softwaremaniacs.org/blog/2011/10/03/core-devs-on-future/en/During recent DjangoCon I caught some of the core developers and asked them to share their views on the future of Django. The idea is hardly original — every leader of something is constantly being pestered about THE FUTURE. But I had a personal interest too. After being away from ...
<p>During recent <a href="http://djangocon.us/">DjangoCon</a> I caught some of the core developers and asked them to share their views on the future of Django. The idea is hardly original — every leader of something is constantly being pestered about THE FUTURE. But I had a personal interest too. After being away from Django development for some time I was wondering where things are going and was also looking for new motivation to jump back on the train.</p>
<p>It's worth noting that all interviews were independent from each other and my interlocutors didn't know of them in advance. I recorded our conversations in an old-fashioned low-tech way with notes on a sheet of paper, so what follows are notes, not their exact words.</p>
<p><a name=more></a></p>
<h2><a href="http://jacobian.org/">Jacob Kaplan-Moss</a></h2>
<p>Jacob is one of the original developers of Django. His dictatorial voice is often the one settling arguments once and for all.</p>
<p><dialog></dialog></p>
<dt>What's next for Django?</dt>
<dd><p>I don't know! There's no general agenda. Django is driven by community, we listen to what people want.</dd>
<dt>What things in Django would you like to see personally?</dt>
<dd>
<p>This is not in any way an "official" position but here are some:</p>
<ul>
<li>replace auth with something better, flexible
<li>Django is not suited very well for real-time web, it'd be good to have it
<li>…and though it's not a personal wish, a NoSQL support seems to be what a lot of people want
</ul>
</dd>
<dt>Will there be a backwards incompatible "Django 2.0"?</dt>
<dd><p>No, there shouldn't be a backwards incompatible release. Some projects tried to break things (notably, Python) but we don't want to do it. Gradual replacement of old things with new ones works for us best.</dd>
<dt>At <a href="http://www.yandex.com/">Yandex</a> we never had a truly backwards compatible upgrade, though…</dt>
<dd>
<p>It's okay to break some things here and there, just not all of them at once. We have deprecation warnings to give people time to change them.</p>
<p>This "not breaking things" is why we have such a good community.</p>
</dd>
<p></p>
<h2><a href="http://cecinestpasun.com/about/">Russel Keith-Magee</a></h2>
<p>Russel is a long-time Django hacker with an area of expertise spanning the whole framework. He's also the president of <a href="https://www.djangoproject.com/foundation/">Django Software Foundation</a>.</p>
<p><dialog></dialog></p>
<dt>What's next for Django?</dt>
<dd>
<p>In general, it's expanding communities. W we should start market to government and enterprise users. This is a job for DSF, it should not be only about Django itself.</p>
<p>Technologically, there are some things to do:</p>
<ul>
<li>using the new Python packaging system more
<li>choosing a Javascript framework to work with
<li>right now Django doesn't do real-time web good but WSGI won't let us redesign the framework for that.
</ul>
</dd>
<dt>Should we start from scratch and make a backwards incompatible Django 2.0?</dt>
<dd>
<p>No. Most of Django core is actually pretty good. Some mistakes were made — like model validation — but mostly it's good.</p>
<p>Python 3 <em>may</em> be the reason for Django 2.0. But backwards compatibility is why we lasted so long. Also, enterprise doesn't like changes. Gradual two-step upgrade process works for us.</p>
</dd>
<p></p>
<h2><a href="http://alexgaynor.net/">Alex Gaynor</a></h2>
<p>Alex is a very (the most?) prolific of core developers, driving scary refactorings of big subsystems. Recently he's also knowing for doing a lot of work for <a href="http://pypy.org/">PyPy</a>.</p>
<p><dialog></dialog></p>
<dt>What's next for Django?</dt>
<dd>
<p>Fix settings!
<p class=note><small>We talked right after <a href="http://blip.tv/djangocon/keynote-glyph-lefkowitz-5573264">Glyph Lefkowitz's keynote</a> where he condemned Django's settings as "a giant global mutable variable where everything happens" and suddenly it became apparent that everyone else hates it too.</small></p>
<p>Also on the list is app refactoring. We probably should look at <a href="http://flask.pocoo.org/">Flask</a>'s app object, at least for inspiration.</p>
<p>Another big refactoring is the template tag API. Which is actually a dependency of template compilation, we should have the new API first.</p>
</dd>
<dt>Should we have a backwards incompatible Django 2.0?</dt>
<dd><p>No, there shouldn't be Django 2.0 Rewriting is forgetting things learned.</dd>
<p></p>
<h2><a href="http://www.aeracode.org/about/">Andrew Godwin</a></h2>
<p>Andrew is the original author of the famous migration tool <a href="http://south.aeracode.org/">South</a>. Working at <a href="https://www.ep.io/">ep.io</a> he is also a guru of massive automated deployment of Django sites.</p>
<p><dialog></dialog></p>
<dt>What's next for Django?</dt>
<dd><p>The core is actually fine, so our main goal is making various components inside Django more useful. "Ajaxy" things are not very important.</dd>
<dt>What's the worst part of the core in your opinion?</dt>
<dd><p>The most problematic part is template system. It's basically the same as it was in 2006 and it needs improvement.</dd>
<dt>Should we have a backwards incompatible Django 2.0?</dt>
<dd><p>No, I feel we could go happily step by step, like we did with "newforms" when they were first introduced with the new name and then changed to just "forms" after some time. Django 2.0 will just occur naturally.</dd>
<p></p>
<h2>Final thoughts</h2>
<figure class=center>
<img src="/media/blog/django-core-devs.jpg" title="Quiz: how many can you name? :-)">
<figcaption>Core developers panel at DjangoCon 2011</figcaption>
</figure>
<p>The most interesting thing for me was that of four people nobody voted in favor of breaking backwards compatibility. And though I didn't have a chance to catch all of the core team I'm pretty sure right now that everyone would say the same. This unanimous opinion is a remarkable thing given that Django now has a dozen core developers instead of a few as just a year ago (IIRC).</p>
<p>Another notable thing is that Django is a fairly stable platform in several senses. In terms of not crashing your site in production it was stable for ages. Then, after 1.0 it became formally stable regarding published APIs. Now it's clear that it won't have any significant architectural revolutions in the foreseeable future either. </p>
<p>Whether it's a good thing or bad depends very much on who you are. If you are a business you can invest in Django and safely plan your hiring strategy for a couple of years. If you're a hacker eager for reinventing the Web — better fork it or start your own framework :-).</p>
<p>And the last but not least, it was really good to talk to those guys :-). Thank you!Django as a micro-framework
2019-08-21T12:07:43.638303-07:00https://softwaremaniacs.org/blog/2011/01/07/django-micro-framework/en/There is a common knowledge that Django is not suitable for writing small ad-hoc projects that don't need anything special apart from displaying some HTML. Boot-strapping a Django project is considered a work substantial enough to not bother with it just for a couple of functions. It was always "obvious" ...
<p>There is a common knowledge that Django is not suitable for writing small ad-hoc projects that don't need anything special apart from displaying some HTML. Boot-strapping a Django project is considered a work substantial enough to not bother with it just for a couple of functions.</p>
<p>It was always "obvious" to me that it's not the case. But recently, due to some phrase I heard from a colleague, I realized that it's not obvious to everyone. Because many people just don't think about it all the time. Well, you know how it happens… So I'd like to show how to create a Django project with minimal hassle.</p>
<p><a name=more></a></p>
<h2>Simplification</h2>
<ol>
<li>
<p>Don't use the <a href="http://docs.djangoproject.com/en/dev/ref/django-admin/#startproject-projectname">startproject</a> command. It creates a nice project base with an annotated settings file but if it's not your first time looking at Django then you can do without it.</p>
</li>
<li>
<p>The manage.py file is also not needed since it's just a local "helper" for django-admin.py.</p>
</li>
<li>
<p>Don't consider the project a container for applications and don't write your code as a pluggable application. Applications are one of the most powerful architectural features of Django but for small tasks you just won't need all this flexibility.</p>
</li>
</ol>
<p>After that we'll have something like this:</p>
<ul>
<li>
<p>settings.py:</p>
<pre><code>DEBUG = True
ROOT_URLCONF = 'urls'
TEMPLATE_DIRS = ['.']
</code></pre>
</li>
<li>
<p>urls.py:</p>
<pre><code>from django.conf.urls.defaults import *
import views
urlpatterns = patterns('',
(r'^$', views.index),
(r'^test/(\d+)/$', views.test),
)
</code></pre>
</li>
<li>
<p>views.py:</p>
<pre><code>from django.shortcuts import render
def index(request):
return render(request, 'index.html', {})
def test(request, id):
return render(request, 'test.html', {'id': id})
</code></pre>
</li>
</ul>
<p>This is a fully functional Django project. You can run it with this command:</p>
<pre><code>django-admin.py --settings=settings runserver
</code></pre>
<p>What you get is a Django request-response pipeline, template system. Most nice things like form library should also work though I didn't check. What you lose is the ORM and all applications depending on it. But hey, it's for a "micro" task after all!</p>
<h2>Extreme</h2>
<p>I wanted to push the idea a bit further to see the moment when it becomes absurd and decided to put everything into <em>one runnable module</em>. And I succeeded:</p>
<ul>
<li>web.py:<pre><code>#### Setup
from django.conf import settings
if not settings.configured:
settings.configure(
DEBUG = True,
ROOT_URLCONF = 'web',
TEMPLATE_DIRS = ['.'],
)
from django.conf.urls.defaults import patterns
urlpatterns = patterns('',
(r'^$', 'web.index'),
(r'^test/(\d+)/$', 'web.test'),
)
#### Handlers
from django.shortcuts import render
def index(request):
return render(request, 'index.html', {})
def test(request, id):
return render(request, 'test.html', {'id': id})
#### Running
if __name__ == '__main__':
from django.core.management import execute_from_command_line
execute_from_command_line()
</code></pre>
</li>
</ul>
<p>You can run the file like this:</p>
<pre><code>python web.py runserver
</code></pre>
<p>However putting it into one file didn't yield much profit. For the file to be readable it has to be separated into named sections. This kind of hints that they really should be separate files after all. Plus, I wouldn't dare to write all this code from memory because of all the boilerplate in it: remember to avoid double initialization of settings, remember long imports etc. </p>
<p>But still it's funny that it worked out :-).</p>
<h2>Afterthought</h2>
<p>When messing with project initialization I've noticed that <code>ROOT_URLCONF</code> setting is used inside the core of request processing. This binds Django pipeline to the existence of a real module with the defined <code>urlpatterns</code> variable. I think Django would be more flexible if it was possible to set URL configuration programmatically with a simple list:</p>
<pre><code>settings.configure(
DEBUG = True,
urlconf = [
(r'^$', index),
],
)
</code></pre>
<p>It might be even useful in practice in cases where you need to setup an environment on the fly. In tests for example.Changing places
2010-12-30T09:17:23.228000-08:00https://softwaremaniacs.org/blog/2010/12/29/changing-places/en/Friday 24 December was my last day at Yandex' Moscow office. As many already know Alena and I have moved to Seattle. She's went to work for Bing and I've got an offer from Yandex to keep working for it as an external consultant which I gladly accepted. Before My ...
<p>Friday 24 December was my last day at <a href="http://yandex.com/">Yandex'</a> Moscow office. As many already know <a href="http://fearlesscoder.blogspot.com/">Alena</a> and I have moved to Seattle. She's went to work for <a href="http://www.bing.com/">Bing</a> and I've got an offer from Yandex to keep working for it as an external consultant which I gladly accepted.</p>
<p><a name=more></a></p>
<h2>Before</h2>
<p>My main job at Yandex was managing a group of developers working on many different projects but all sharing one technological platform: we create web services using Python. Another, also important, part of my job was maintaining and developing a stack of Python-related technologies: Django, various tools, build process, documentation. And I have to admit that this latter part was always sacrificed in favor of the former.</p>
<h2>After</h2>
<p>Changing place of residence means that I will no longer be able to do any personal management. And that additional load just becomes my main job. Which, by the way, I like very much. Django-related stuff at Yandex needs some tidying up. I'd like to bringing it up to modern standards, automate routine infrastructure tasks of deployment, testing, monitoring…</p>
<p>One other thing that I don't want to give up is various APIs to Yandex services. None of them has become a hit yet but I have a couple of ideas on how to fix it. Documentation, feature extension, creating of exemplary apps demonstrating APIs best parts — all this is much needed.</p>
<h2>Blogging</h2>
<p>I hope that now I'll have more time for technical blogging. Doing it at an office open space was absolutely impossible, I was mostly exhausted after-hours and weekends are full of house work. This has lead to many interesting things sitting for years as drafts, acquiring dust. Though it might be just because of my laziness. Let's see who will win this time :-).</p>
<p>While I'm at it here's another thing. My friends keep poking me about writing something generally life-related. Doing it here would be weird so I'm trying to pick up some sane way of doing it. I will be writing some text, linking pictures from my photo hosting and I'd like to be able to forward there my content from other places. Current options are:</p>
<ul>
<li>
<p><a href="http://www.facebook.com/">Facebook</a>. It looks like it's made exactly for this and about "everyone" is now on it. What stops me is unwillingness to put a pretty large part of my life into hands of its founder who seems to think that he knows better on how everyone else should live. Besides the very idea of a "single social network for everyone" is <a href="http://www.slideshare.net/padday/the-real-life-social-network-v2">flawed</a> so I don't have a goal to collect all my friends on Facebook.</p>
</li>
<li>
<p><a href="https://twitter.com/isagalaev">Twitter</a>. I tweet and I'll keep doing it. However it's just not the right format for life notes that I'm about to do.</p>
</li>
<li>
<p><a href="http://www.tumblr.com/">Tumblr</a>. It's a completely dark horse to me right now. It kind of fits the "Twitter, but bigger" description but I don't know if it can actually live up to my needs. </p>
</li>
</ul>
<p>Any advice?Marcus: a bilingual blog
2012-10-20T13:08:05.100000-07:00https://softwaremaniacs.org/blog/2010/07/18/marcus-bilingual-blog/en/Back in January I quietly replaced here a WordPress installation with a custom blog software. I didn't write anything about it then partly because of laziness and partly because of the fact that the software itself was quite banal. However one of the advantages of having a custom solution is ...
<p>Back in January I quietly replaced here a WordPress installation with a custom blog software. I didn't write anything about it then partly because of laziness and partly because of the fact that the software itself was quite banal.</p>
<p>However one of the advantages of having a custom solution is the ability to implement new features in exactly the way you want them. So recently after implementing support for bilingual content I decided that it's now worth giving it a technical overview.</p>
<p>Let me clarify that this software isn't intended for general consumption. And though it is written in a stand-alone pluggable fashion its feature set is not likely fit most bloggers' needs.</p>
<p><a name=more></a></p>
<h2>Code</h2>
<p>The code is available in a <a href="http://bazaar.launchpad.net/~isagalaev/%2Bjunk/marcus/files">branch at Launchpad</a>. It's not big, just about 860 lines, and in my opinion is quite readable. What's interesting is that bilingual stuff accounts for about 250 lines of those. Anyway the blog itself is so small because many things are extracted in separate libraries:</p>
<ul>
<li><a href="https://launchpad.net/pingdjack">pingdjack</a> is used to send and accept pingbacks</li>
<li><a href="https://launchpad.net/scipio">scipio</a> covers OpenID authentication in comments and implements anti-spam pipeline</li>
<li><a href="https://launchpad.net/subhub">subhub</a> implements a personal <a href="http://code.google.com/p/pubsubhubbub/">PSHB</a> hub which allows feed readers to receive and update my posts instantly (though some of them haven't yet catch up... <a href="http://www.google.com/reader">Google Reader</a>, I'm looking at you)</li>
</ul>
<h2>Features</h2>
<p>The blog doesn't have much special in it. I've implemented everything I was actually using in WP including some things covered by plugins. This is why nobody has noticed the switching: everything works the same. Except for small bugs that were fixed along the way: typography is no longer applied to code snippets, navigational links don't lose page numbers, etc.</p>
<p>All editing is done in Django admin. The interface is <em>radically</em> simpler than the old WP-style "we-have-the-best-blogging-system-look-how-many-things-we-got" dashboard. The only interesting admin customization is a <a href="http://bazaar.launchpad.net/~isagalaev/+junk/marcus/annotate/38/admin.py#L14">FilterSpec</a> for fields of type "boolean time". Essentially they are pretty normal <code>DateTimeField(null=True)</code> that work both as flags and as time values. A typical example is "published" field for an article: while it has None the article is considered a draft, when it gets a specific time value it becomes published.</p>
<p>The hardest thing to implement were feeds. Though Django does support them out of the box it does this not exactly in the way I need it. I plan to write another post about it, there are a couple of things to grumble about :-).</p>
<h2>Bilingual internationalization</h2>
<p>As the first principle of my bilingual design I decided not to implement an abstract universal support for arbitrary content in multiple languages. This is quite hard to implement and even harder to implement it efficiently. Instead I've just framed my problem into two limited usecases and kept them in mind all the time:</p>
<ul>
<li>Current audience of this blog can read Russian and, to a lesser extent, English and should get posts in both languages with a preference for Russian version.</li>
<li>English speaking audience generally has no use for Russian content, so there should be an English-only version of the blog.</li>
</ul>
<p>Obviously not all readers fall into these categories but since all the content is anyway available I decided not to complicate neither code nor UI with support for corner cases.</p>
<h3>Models</h3>
<p>Models that work with bilingual content store it explicitly in separate fields for both languages:</p>
<pre><code>class Article(models.Model):
title_ru = models.CharField(max_length=255, blank=True)
text_ru = models.TextField(blank=True)
title_en = models.CharField(max_length=255, blank=True)
text_en = models.TextField(blank=True)
# ...
</code></pre>
<p>Granted, it wouldn't be very convenient to work with those fields directly because it would require having conditions all over the code checking which of the fields are available. This is helped as follows.</p>
<p>A model has methods that return content depending on the language passed into them:</p>
<pre><code>class Article(models.Model):
def title(self, language=None):
# return title_ru or title_en
title.needs_language = True
def html(self, language=None):
# format text_ru or text_en
html.needs_language = True
def get_absolute_url(self, language=None):
# generate URL depending on the language
get_absolute_url.needs_language = True
# etc.
</code></pre>
<p>I'll explain implementation logic of these methods later. What's interesting now is the attribute "needs_language" that is used by a specialized proxy <a href="http://bazaar.launchpad.net/~isagalaev/+junk/marcus/annotate/96/models.py#L21">Translation</a>. Its job is to pass implicitly the language of translation into all methods that require it. The proxy is used then to wrap all bilingual objects before they can be used in Python or templates code to make it possible to write just <code>{{ article.title }}</code> and have it automatically translated into <code>article.title(language)</code> call.</p>
<pre><code>class Translation(object):
def __init__(self, obj, language):
super(Translation, self).__init__()
self.obj = obj
self.language = language
def __getattr__(self, name):
attr = getattr(self.obj, name)
if getattr(attr, 'needs_language', None):
attr = curry(attr, self.language)
return attr
</code></pre>
<p>To make it possible to "translate" objects (i.e. wrap them in proxies) right in template code I've also made a <a href="http://bazaar.launchpad.net/~isagalaev/+junk/marcus/annotate/118/templatetags/marcus_tags.py#L83">filter "translate"</a>. It is smart enough to accept single objects, flat sequences of objects and even trees represented as nested sequences:</p>
<pre><code>{% with comment.article|translate:language as a %}
...
{% for cat in article.categories.all|translate:language %}
...
{% tree object_list|astree:"parent"|translate:language %}
</code></pre>
<h3>URLs and language selection</h3>
<p>Having divided the audience into two categories I needed different URL schemes to address them. I ended up with a simple solution where URLs for English content got "en/" part at the end. In this case Russian content isn't shown at all. Without the "en/" all the content is shown with a preference for Russian version when it's available. As a side effect I've also got working URLs ended with "ru/" which yield only Russian content. But they are not advertised anywhere in UI.</p>
<div class=note><small>
<p>Here is, by the way, where the decision to keep language specific fields inside models instead of having them in separate tables turned out to be good: queries tend to be very simple. For example querying articles available only in English is as simple as filtering them without any additional joins:</p>
<pre><code>Article.objects.exclude(text_en='')</code></pre>
</small></div>
<p>Such URLs provide three different values for a language in which a user requests the content: "en", "ru" and None. The content itself also can be available in three variants: in Russian , in English and in both languages (technically there's a fourth variant: when content isn't available in any language but it's not of much practical interest :-) ). Therefore all language-aware methods should return proper content depending on these two parameters.</p>
<p>Logic is not complicated but requires a bit of attention in corner cases. For example method Article.title looks like this:</p>
<pre><code>def title(self, language=None):
if language:
return self.title_en if language == 'en' else self.title_ru
else:
return self.title_ru or self.title_en
</code></pre>
<p>In the situation when a caller requests an English title of a purely Russian article it will get an empty string. I decided not to raise any special Exception in this case because this situation is prevented at the application level where objects that don't have required content are filtered out early.</p>
<h3>Translation of user interface</h3>
<p>All the infrastructure for translating user interface is available in Django's <a href="http://docs.djangoproject.com/en/dev/topics/i18n/">internationalization system</a>. However what I didn't use from it was a middleware that detects language based on user settings. Since I get language from URLs I initialize translation manually in each view function that works with languages:</p>
<pre><code>translation.activate(language or 'ru')
</code></pre>
<p>The <code>or 'ru'</code> part means that by default the language of user interface is switched to Russian.</p>
<p>Because of the concept of a default language I couldn't use Django's built-in variable LANGUAGE_CODE in templates because it always set to some particular language. Instead I pass my own language variable into templates wherever it's needed.</p>
<p>Some people may point out that repeating all these lines in several views violates <a href="http://docs.djangoproject.com/en/dev/misc/design-philosophies/#don-t-repeat-yourself-dry">DRY</a> (shockingly!). And that's true. But here I deliberately refused the temptation to write some generalized code because I suspect that all the fuss with middleware and context processors just won't worth the benefit.</p>
<h2>Name</h2>
<p>I named this blog software "Marcus" after <a href="http://en.wikipedia.org/wiki/Marcus_Antonius_%28orator%29">Marcus Antonius (orator)</a>. This is not that famous general who had affairs with Cleopatra. This one was known by his good memory that allowed him to remember and pronounce in courts thoroughly prepared speeches that still appeared as if he spoke impromptu.</p>
<p>I think that this is the most suitable Ancient Roman name for a blog :-).