[Python-il] Statically analyzing a Python module to find assignments

cool-RR cool-rr at cool-rr.com
Sat Apr 16 13:26:10 IDT 2011

On Sat, Apr 16, 2011 at 12:20 PM, Tal Einat <taleinat at gmail.com> wrote:

> cool-RR wrote:
>> Hello folks,
>> I want to let people who write GarlicSim simpacks write an `__init__.py`
>> module like this:
>>     '''
>>     This is the simpack's docstring.
>>     Bla bla bla.
>>     '''
>>     name = 'Name of the simpack'
>>     tags = ['list', 'of', 'simpack-tags']
>>     __version__ = '0.9.3'
>> And I want GarlicSim to be able to get all of these things: The docstring,
>> the name, the tags and the version. And I want to show them in this
>> widget <http://dl.dropbox.com/u/1927707/SimpackSelectionDialog.png>. But
>> I want to do it without importing the module, because `__init__.py` might
>> import the entire package which can be heavy. (I want to do this process for
>> many heavy simpacks.)
>> We can assume that the user doesn't do anything computationally trickier
>> then a simple literal assignment.
>> I heard that the `ast` module does things like this. But, I also want to
>> be able to do this process on compiled files, e.g. `__init__.pyc` or
>> `__init__.pyo`, and I don't know how to do this with the `ast` module.
> Why on earth would you want to do this on .pyc or .pyo files? Do you want
> to support source-less simpacks?

Yes, definitely. For example, py2exe creates distributions with compiled
files only. Some people like that in order to keep their source code
somewhat secret. Some people, like me, don't like it but still use it
because neither py2exe nor bbfreeze let you turn that feature off.

If you can ensure that __init__.py is available, you can pick the lines from
> the file which include the doc-string and assignments quite easily, and
> execute them in a dummy environment to get the values. You can fall back to
> importing the package if this fails.

Not good enough. I really want to support .pyc files and I really want
to not import all the simpacks.

Mangling __import__ temporarily might work, sounds like a hack I would
> prefer to avoid. And theoretically __init__.py can include other heavy code.
> And you could break stuff if __init__.py assumes that the imports work in a
> way that returning a mock object breaks.

Definitely not pretty. But can you come up with an example of how it would
be broken? I think most such examples would be pretty contrived and not
something that's likely to be written in real life.

> Do you have a really good reason not to make the writer of a simpack
> responsible for making __init__.py quick to import? Then you can optimize
> only if a real need arises.

I personally believe that `__init__.py` should import important stuff from
the package, and not only in simpacks but in general in Python packages.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://hamakor.org.il/pipermail/python-il/attachments/20110416/0d031beb/attachment.htm>

More information about the Python-il mailing list