API docs

Update: mwh gave me commit access and the HTML-generation stuff is now in SVN. Go for your lives!

API doc generation in Python has been a foul pit of despair for a while. You wouldn't think so, at first glance: there are seemingly great tools such as Pydoc, Epydoc, and so forth. But if you try to use them with modern Python toolkits (such as zope.interface) and on large codebases, they're almost impossible to work with: Twisted has been using Epydoc for years, and we have a massive set of monkeypatches just to get it run over our code at all. Since Twisted 2.0, it's required a blood sacrifice for every run (which I have been leaving to others to do).

A week or three ago, mwh wrote a thing that walks a tree of Python source and uses the AST module to generate a nice structure representing all things Doccy about it. The other night, I wrote a chunk of code to convert that structure to HTML. You can browse the Eighty One Megabytes of Twisted API I generated with it. It's pretty lame looking, but it's a start. And it doesn't take 15 minutes and eat a gig of RAM during generation! Hooray.

I'd really really like to, instead of writing HTML directly, write to Fredrik Lundh's Pyref format, and then use his tools (if he ever releases them!) to generate HTML that will look like whatever format he comes up for the Python standard library docs.

I hope mwh doesn't mind my giving out the link to his public repository, but oh well! the code can be checked out from the following URL via SVN:

http://codespeak.net/svn/user/mwh/docextractor

The HTML-generating stuff isn't there yet, but hopefully it will be soon.