Dicts are now ordered, get used to it



There were several moments over the last few weeks when I heard people discuss differences between Python lists and dicts and one of the first ones mentioned was that lists are ordered and dicts are not. Well, not anymore. Quoting the docs referenced above: Changed in version 3.7: Dictionary order ...

Python stdlib gems: collections.Counter



Here's a toy problem. Given a corpus of phone numbers for different countries determine a most prevalent display format in each country and use it to re-format an arbitrary phone number for its country. For example, if most US numbers in our data corpus are written like xxx-xxx-xxxx then the ...

ijson 2.0


Python, My software

Yesterday I released version 2.0 of the streaming JSON parser ijson. It mostly includes bug fixes accumulated over the last year and the only reason to change the major part of the version number was that import ijson doesn't do any discovery magic anymore. Import Previously, when you did import ...

Refactoring discovery protocol

It's been a while since my last update on the python3-openid refactoring. Though I still work on it pretty actively, I totally failed at documenting the process as I planned in the beginning. So I came up with a new plan. New plan First of all, I admit to gravely ...

Carpet testing

Carpet bombing is a "large aerial bombing done in a progressive manner to inflict damage in every part of a selected area of land." Similarly, carpet testing is done by progressively tossing random data samples at your code without regard for its internal structure, hoping that sufficient amount of data ...

Dissecting fetchers

This is the first installment of my diaries on refactoring python3-openid. The post is turning out pretty big so may be I should try doing them more often. Warm-up I started with fixing failing tests, because you can't do refactoring without tests. The root cause of errors was somewhere inside ...

OpenID library for Python 3

Apparently, there is no obvious choice for an OpenID library in Python 3. In Python 2 there's python-openid which — despite being an ugly over-engineered mess of a code — works and conforms to the spec. Unfortunately, the same can't be said about its independent port to Python3 python3-openid, which ...

Memory is slow

Did you know that memory is slow compared to CPU? I kinda knew too but recently I've got a revelation from two unrelated sources about how it affects design of programming languages. I also learned about a new thing called "value/reference type dichotomy". I've stumbled upon this term while reading ...

PyCon Russia 2014



To my surprise this blog is still read by people whose profession involves Python. To all of them I'd like to remind that the second Russian PyCon starts in just two weeks and you might want to consider going. There'll be smart and important people abound and, if my experience ...

Glyph Lefkowitz on threads


Python, Software design

I value very much the ability to put complex concepts into words in a systematic manner. And I thrive to do the same (at least, I did try when I was blogging actively). So now I'm a big fan of Glyph who laid out everything that is problematic about threads: ...

ijson 1.0



I've finally scraped some time to finish and release a new version of ijson — 1.0. New stuff: support for YAJL 2 pure Python parser compatibility with Python 2 and Python 3 Parsing improvements On this I have already posted in details not that long ago. To summarize, ijson now ...

ijson on PyPy, Episode 3: New parsing



It's a funny thing when after neglecting your project for a year you get a question on whether it's orphaned and then suddenly you find yourself hacking on it for few days straight… Knowing that your work is needed and appreciated is the greatest motivator! Anyway… The news I wanted ...

When to use decorators in Python



The @decorator syntax in Python is easy to abuse. After all, it's simply a syntactic sugar for: obj = decorator(obj) The obj must be a function or a class but Python doesn't care about the output value that is then assigned to the same name. It may be, quite literally, ...

virtualenv: solved



My problem with virtualenv is solved. Great thanks to Malcolm Tredinnick for taking the time to understand it and pushing me into the right direction. As I said in the previous post, I didn't really need the isolation feature of virtualenv with my single-user, single-codebase site. What I really needed ...

SM.Org software update 2012


Python, Ubuntu, Django

Over the course of a few recent weeks I updated this site to a more modern software and revised some previously made choices. This one was loooong overdue considering that I still ran Ubuntu 9.10 before the update meaning that the system was almost 3 years old. Here are some ...

ijson on PyPy, Episode 2: Warm-up



Today I've come upon a very interesting development in the story of optimizing pure Python version of ijson. The thing as I left them yesterday were like this: Original yajl wrapper0.47 sec CPython0.84 sec PyPy1.30 sec These are the times of parsing a JSON array of 10000 objects. The parser ...

ijson on PyPy



I happen to follow Alex Gaynor on Twitter and his ravings on speed and general awesomeness of PyPy have inspired me to a small experiment. I have this iterative JSON parser — ijson — which is a ctypes-based wrapper around C library yajl by Lloyd Hilaiel. I don't know of ...




The usual solution to parsing JSON in Python is using either the included in the standard library simplejson or the third-party cjson that became popular recently. Both libraries process JSON in one piece: parse the whole thing and return a native Python object. However there is a certain value in ...

elementflow: a formal introduction



Last Monday during Moscow web developer conference DevConf I was giving a master class where I presented a Python library for streaming XML generation. Upon receiving a much-hoped feedback, today I'd like to formally present the project and its first release. Summary The release 0.1 can be downloaded from PyPI ...