Great tool, but not to be put in the wrong hands. The folks at Django got it right. We used it for unifying, simplifying and ensuring consistency of deeply nested configuration data entered using different means. However, in environment with less experience, I have seen "Akira-final-scene-tetsuo" monsters come to life with this.
Another loosely related suggestion: if we could stay away from singleton as examples, I think we would save ourselves a lot of future troubles. ^^
"Great tool, but not to be put in the wrong hands."
I wouldn't quite say this - I wouldn't want to discourage someone who's going to learn them to go ahead, but I'd just say "Only use them when you really need to use them and if you're going to use them make sure you really understand them".
The problem with this is the "wrong hands" are exactly the kind of programmers who are convinced they really need complex abstractions when a much simpler option is available.
I enjoyed reading through the Django ORM code to understand how it worked. Metaclasses in python are fun, but I can rarely justify actually using them for anything other than messing around. Oh but how I wish I needed to use them more!
We use meta-classes extensively. One of the simpler examples I have used are as follows:
We have a class that has different methods that performs numerical analysis/computations of financial models. These methods need to be tested with different subsets of data. The number of combinations of tests and test-data are huge, whereas the 'testing code' is simple and repetitive. Having these tests in a loop is not useful due to way test errors get reported.
Therefore, we generate a test class using the meta classes, that creates 'test_*' methods that are eventually executed. With such methods in place, we have around 100 tests generated within less than 100 lines of code, without loosing the detailed error reporting in case of failing tests.
Could the same thing be achieved by subclassing your base test class and then overriding setUp to inject the correct data for that combination? I'm pretty sure I'm oversimplifying, but I'm really interested in your method.
In Django, when we add a new field type, you need to remember to test all the integrations (admin, forms, migrations, etc) and that's not always done correctly. It'd be great if there were some easier method than just subclassing a template test class and implementing all tests from the ground up.
I do not think it would work with sub-classing. With the approach we have it is just easier to add the 'new' method name to test into the list argument of the meta class. The meta class would generate a `test` method with the passed in function name.
With the subclass approach you are required to implement a new class and essentially write repetitive code, or reuse existing functions. Reusing existing test methods does not provide good error reporting as I had stated earlier.
It is late in the day. Shortly, I will try to post a gist example of what I am talking about, if you are still interested.
This enables things like plugin systems, where you want the act of importing a plugin which defines some objects to load those objects into the available plugin list.
I've written a few code spikes using metaclasses and to be honest, it never worked out well for me. I always found a simpler, clearer way of achieving the same outcome without them and the code that I did write felt dirty.
It's a very powerful language feature. Also very dangerous. It's a magnet for technical debt and obscure bugs.
I've used them to build a serialization/deserialization library for a proprietary protocol. To cut down on needless boilerplate I had the metaclass build setter/getter/en- and decoders from a static member variable. Inspired also by how Django describes its data model.
I like Ruby's object model and uniform access principle much better. You don't have to worry about descriptors and meta-classes when doing metaprogramming. Just a few hook methods and `instance_eval` and you're good to go.
Caveat: I haven't written any Perl except for an occasional one-liner in a decade.
But I do a fair amount of Python, and a 20 minute cruise through the Moose docs leads me to believe that the most similar thing to Moose in the Python world is, well, Python.
Rather than having the kind of parametric polymorphism you get with roles, Python prefers duck typing. And for the practical use of adding some composed, reusable, but orthogonal state and behavior to a class, Python provides multiple inheritance. For type-checking issues where duck typing is not sufficient, Python has Abstract Base Classes.
Now, I am aware that roles are not the same thing as duck typing or multiple inheritance, or ABCs. Not having roles in Python is a design decision. http://legacy.python.org/dev/peps/pep-3133/
But I still think in terms of the ability to use the language to write clean OOP code, Python is closer to Perl5 + Moose, than Perl5 alone is to Perl5 + Moose.
(Not a knock on Perl5 or Moose, between 2002 and 2005, I did a bunch of web and systems programming in Perl, and I wish Moose existed then.)
This topic has been beat to death. I guess there are more articles explaining metaclasses then people who understand them. There are 11K results for /understanding python metaclasses/ in google.
Another loosely related suggestion: if we could stay away from singleton as examples, I think we would save ourselves a lot of future troubles. ^^