[Python-il] Fwd: Is there a Python module for handling Python object addresses?

Amit Aronovitch aronovitch at gmail.com
Sat Sep 11 22:07:19 IDT 2010


On Sat, Sep 11, 2010 at 6:04 PM, cool-RR <cool-rr at cool-rr.com> wrote:

> Hello Amit,
>
> On Sat, Sep 11, 2010 at 4:43 PM, Amit Aronovitch <aronovitch at gmail.com>wrote:
>
>> Hi Ram,
>>
>> It seems to me that the reason you are not getting an answer is that
>> people do not understand your question.
>> I, for one, do not understand what you mean by "address".
>> The common interpretation of this term is the physical (virtual) memory
>> address of the PyObject (which might be useful for mangling with ctypes or
>> an extension), but from your comments it does not seem that this is what you
>> want.
>> I assume you know of id(), so probably you do not need that as a unique
>> id.
>>
>
> I defined "object address" in the beginning my first message. In case it
> wasn't clear enough, I will give two examples:
>
> 1. Here's a little shell session:
>
> >>> import re
>
> >>> re.match('q', 'q')
>
> <_sre.SRE_Match object at 0x00D38250>
>
>
> Do you see that "_sre.SRE_Match" thing? That's what I mean by address. You
> can easily understand which class the object is just from this string.
>
>
Some common terminology (useful for clearing up misunderstandings):

_sre.SRE_Match is the *type* of the match object.
0x00D38250 is the physical address (or perhaps a hash of the address) of the
object:

>>> import re
>>> r = re.match("q","q")
>>> r
<_sre.SRE_Match object at 0x226ee00>
>>> type(r)
<type '_sre.SRE_Match'>
>>> hex(id(r))
'0x226ee00'

The string "_sre.SRE_Match" is the (fully qualified) name of the type of r

>>> type(r).__name__
'SRE_Match'
>>> type(r).__module__
'_sre'

2. Here's a statement from a Django settings module:
>
> INSTALLED_APPS = (
>
>     'django.contrib.auth',
>
>     'django.contrib.contenttypes',
>
>     'django.contrib.sessions',
>
>     'django.contrib.sites',
>
> )
>
> Those strings are addresses. They tell Django which class you are referring
> to, without actually having to import the class.
>
>
>
These are (fully qualified) module names.
Distinction: In this case the names are the names of the module objects
themselves.
In the previous case it was the name of the *type* of the object.

I know that this definition of "object address" is my own idiosyncratic
> definition, but I think it will be natural to Python programmers, since
> Python (and Django) use this concept, except they don't explicitly call it
> an address.
>
> Please specify a clear use case.
>>
>
> 1. Django settings module.
> 2. `repr`. Especially if you want to shorten the address, according to
> point 3 of my first message. It's a lot nicer to have:
>
> <garlicsim.Project containing 101 nodes and employing 4 crunchers
> at 0x1f668d0>
>
>
> Then to have
>
> <garlicsim.asynchronous_crunching.project.Project containing 101 nodes and
> employing 4 crunchers at 0x1f668d0>
>
>
> (Since `Project` is already available in the `garlicsim` namespace.)
>

The fully qualified name is the only one you can find in O(1), because this
is the information which is stored in the object.
To find other modules which import it, you'd have to search, and also the
result might not be constant.
e.g.

>>> from django.contrib import auth as a
>>> a.__name__
'django.contrib.auth'
>>> import sys
>>> sys.auth                   # auth is NOT available in module sys
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'auth'
>>> sys.auth = a             # from now on, it is
>>> sys.auth.__name__   # but the name stays the same
'django.contrib.auth'

If you want to shorten the name that appears in the repr, you can always
change the __name__ (or __module__ in case it is a class), in your
garlicsim.__init__, right after you import it (however, this might confuse
debuggers etc. that need to find the source file in which your
class/function/module was defined).

>>> a.__name__ = "kuku.auth"
>>> sys.auth
<module 'kuku.auth' from
'/var/lib/python-support/python2.6/django/contrib/auth/__init__.py'>



> 3. I'm making a GUI in which the user will be able to manipulate Python
> functions, and I want to display the shortest name possible on the screen.
>

I think you'd have to do a nasty search in sys.modules.
I'm almost certain I wrote code that does something like that, but that was
ages ago and I do not have access to it any more. Sorry.
Note that if you want your GUI to be able to find the *source file* where
the object is defined, the original fully-qualified-name
would be more useful.


>
>> On Fri, Sep 10, 2010 at 9:40 PM, cool-RR <cool-rr at cool-rr.com> wrote:
>>
>>> On Fri, Sep 10, 2010 at 8:23 PM, Imri Goldberg <lorgandon at gmail.com>wrote:
>>>
>>>> Several points:
>>>> 1. Unless you are doing something very special (like a debugger),
>>>> needing this seems like a bad design choice. Just my shnekel.
>>>> 2. Although you can (in some cases for some value of some) define it,
>>>> talking about an "official address" of an object, seems extremely
>>>> non-pythonic.
>>>> 3. To your actual question: you can use the gc module to get the
>>>> referrers to a specific object. By traversing this tree structure, you can
>>>> probably extract the information you want. Furthermore, for many objects,
>>>> such as instances and functions, finding the definition point is not that
>>>> hard (using the inspect module). In any case it seems to me that what you
>>>> seek to do seek to do will be very hard to accomplish correctly, require
>>>> some "deep magic", and IMHO, yield low returns.
>>>>
>>>> Cheers,
>>>> Imri
>>>>
>>>
>>> Hi Imri,
>>>
>>> I said:
>>> "When I say "object", I mostly mean classes, modules, functions, methods,
>>> stuff like these, which usually do not change their address."
>>>
>>
>> No object in Python ever changes its physical address (at least I can not
>> think of any such scenario).
>>
>
> We are not speaking about the physical address in memory.
>
>
>>
>>> Why would I need to use gc for it? I can get the address for almost any
>>> object with `__module__`.
>>>
>>
>> If you need the address for enumerating the referrers, the answer would
>> indeed be gc.
>>
> Do you mean you want the container object, or the source file where the
>> object is defined?
>> Maybe inspect.getfile() or inspect.getmodule() could help?
>>
>
> I assume that now when I explained my "address" definition more clearly,
> it's understood that `gc` and `inspect` are not needed.
>
>
Maybe inspect might still be relevant  - note that help() uses it to display
some information.
It has been more useful with older versions of Python, because it had a more
consistant interface accross different types than getting the information
directly from the objects (e.g. functions did not have a __name__, and you
had to use func_name, for old-style instances you had to use __class__
rather than type() etc.)

Note that some types of objects just do not contain this information (ints,
>> strings etc.)
>>
>
> I said:
>
> "When I say "object", I mostly mean classes, modules, functions, methods,
> stuff like these, which usually do not change their address."
>
>
>
>>> This kind of thing is used in Django settings module, (specifying strings
>>> instead of objects themselves,) so why is it a bad design choice when I want
>>> to do it?
>>> Also, the pickle module in the standard library uses an object's address
>>> to pickle some kinds of objects (classes, functions).
>>>
>>>
>> pickle uses the name of the module and the name of the function/class to
>> pickle it (assuming the modules would be available when unpickling - if not,
>> you'd need the "fickle" module mentioned in a previous post).
>>
>
> Well, that is exactly the address.
>
>

OK. As I said, I think "fully qualified name" would cause less
misunderstandings :-)


>>  Anyway, I'm frustrated from getting a lot of "You're doing it wrong"
>>> answers for this question, and I just want an answer, not explanation on why
>>> I shouldn't do it. (My question is, "Is there a mature module for these
>>> tasks?")
>>>
>>>
>> If I understand correctly, "inspect" is the module you seek.
>> But to get a better answer, you will have to give a clearer description of
>> what you are trying to achieve.
>>
>>     AA
>>
>
> I hope that I've given a clear enough description in this message.
>
>
Now that everything should be clearer, I hope someone else can come up with
a more useful answer than mine (should search sys.modules, lost the code
that does something like that...)

Anyway, I already started coding it, I'd say it's 20% complete now.
> ("Complete" being defined as "ready for alpha or beta release, with tests
> and documentation.")
>
>
Good luck!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://hamakor.org.il/pipermail/python-il/attachments/20100911/2d448bd9/attachment-0001.htm 


More information about the Python-il mailing list