[Python-il] [OFFTOPIC] Two Python-IL mailing lists? (was: Re: Apache with mod WSGI (for django) crashes when you "import nltk")

Omer Zak w1 at zak.co.il
Mon Oct 14 21:23:49 IST 2013


Seems that there are two Python-IL mailing lists:
python-il at googlegroups.com
python-il at hamakor.org.il
I get messages sent to the @hamakor.org.il list.

On Mon, 2013-10-14 at 15:02 -0400, Avishalom Shalit wrote:
> btw python-il is rejecting my post, so if anyone wants to forward it
> there feel free
> 
> 
> 
> -- vish
> 
> 
> 
> 
> On 14 October 2013 14:59, Avishalom Shalit <avishalom at gmail.com>
> wrote:
>         well, it was solved by someone else, and it was a while ago, 
>         
>         for our needs we run it not under apache, but under > python
>         manage.py runserver 0:8000
>         
>         and it works fine. 
>         
>         
>         i think at some point we were able to get it running under
>         apache after updating our linux distribution and apache, 
>         but we fell back to the django dev server
>         
>         
>         -- vish
>         
>         
>         
>         
>         On 14 October 2013 14:26, <gritchie at gmail.com> wrote:
>                 Hi Vish - Did you manage to get this working somehow?
>                 I am also having problems using the NLTK in a django
>                 app. I'd appreciate any tips you have on setting
>                 things up correctly.
>                 
>                 
>                 Thanks,
>                 
>                 
>                 Graham
>                 
>                 
>                 On Friday, February 1, 2013 10:49:00 AM UTC, Avishalom
>                 Shalit wrote:
>                         thanks. 
>                         
>                         actually this is an internal app, only
>                         available on our VPN, 
>                         so security is not an issue , 
>                         and i only expect a maximum of 4 users 
>                         
>                         
>                         i will look at the other setups. 
>                         thanks
>                         
>                         
>                         -- vish
>                         
>                         
>                         
>                         
>                         On 31 January 2013 23:48, Emanuel Ilyayev
>                         <emi... at gmail.com> wrote:
>                         
>                                 I don't know enough NLTK but I work
>                                 with django :)
>                                 
>                                 
>                                 From Asaf's description it looks like
>                                 you have to change your architecture.
>                                 Apache - in it's default configuration
>                                 - is not efficient in working with
>                                 heavy processes because it creates a
>                                 new process for each request. There
>                                 are better setups like using gUnicorn
>                                 or uWSGI that load n workers and
>                                 distribute the work between them
>                                 (usually n = number of cores X 2 + 1).
>                                 
>                                 
>                                 More robust and scalable setup would
>                                 include a separate workers that answer
>                                 to the NLTK requests asynchronously
>                                 and django approaches these workers
>                                 via a message queue. This setup will
>                                 allow you to put your NLTK workers
>                                 even on a separate machine without
>                                 creating situation where your web
>                                 server is competing with your NLTK
>                                 workers on limited resources (CPU and
>                                 RAM).
>                                 
>                                 
>                                 Even if you will eventually find the
>                                 way to configure apache to load NLTK
>                                 without crashing - the URL that
>                                 handles NLTK requests would be a
>                                 perfect point to attack you server and
>                                 to bring it into a DOS (denial of
>                                 service) situation using only a couple
>                                 of strong machines approaching this
>                                 URL....
>                                 
>                                 
>                                 I urge you to read a little bit about
>                                 gEvent and Celery to understand what
>                                 I'm talking about.
>                                 
>                                 
>                                 HTH
>                                 
>                                 
>                                 --
>                                 Emanuel
>                                 
>                                 
>                                 
>                                 
>                                 
>                                 
>                                 On Thu, Jan 31, 2013 at 7:30 PM, asaf
>                                 greenberg <asafgr... at gmail.com> wrote:
>                                 
>                                         
>                                         i don't know enough django,
>                                         but i worked with nltk.
>                                         NLTK is a very heavy module,
>                                         lagging on import is expected,
>                                         especially if you're using
>                                         certain modules.
>                                         
>                                         AFAIK you should `import' it
>                                         only once, on server
>                                         (re)start, and it costs about
>                                         10-30 secs (did you optimize
>                                         with *pyc or *pyo?). unless
>                                         you're short on RAM... but i
>                                         hope that's not the case.
>                                         
>                                         NLTK has also many
>                                         sub-modules, which can and
>                                         should be disabled, for
>                                         performance.
>                                         
>                                         Does it hang elsewhere (apart
>                                         from server startup)?
>                                         does it have a longer delay
>                                         than 20-30 secs.?
>                                         
>                                         
>                                         
>                                         On 1/31/2013 6:44 PM,
>                                         Avishalom Shalit wrote:
>                                         
>                                         > As title. 
>                                         > 
>                                         > 
>                                         > It just silently hangs. 
>                                         > 
>                                         > 
>                                         > as far as i found on google,
>                                         > other people have ran into
>                                         > it, 
>                                         > 
>                                         > but nobody posted a
>                                         > solution. 
>                                         > 
>                                         > 
>                                         > anybody overcame this
>                                         > before ?
>                                         > 
>                                         > 
>                                         > thanks
>                                         > 
>                                         > 
>                                         > -- vish

-- 
Make love not war.
More cleavage, less carnage.
My own blog is at http://www.zak.co.il/tddpirate/

My opinions, as expressed in this E-mail message, are mine alone.
They do not represent the official policy of any organization with which
I may be affiliated in any way.
WARNING TO SPAMMERS:  at http://www.zak.co.il/spamwarning.html



More information about the Python-il mailing list