KDS Software Group » testing django with twill and nosetests

testing django with twill and nosetests

Posted by Andrey Khavryuchenko on 04 Jun 2007 at 09:17 pm | Tagged as: Blog, django

Just a little writeup..

Now have a pleasure of not starting django development webserver, nor using apache to do http-level tests in my project.

Thanks to twill and its wsgi integration.

Works smoothly now, but caused me little cursing when twill refused to see rss feed.

After digging the twill code I’ve changed a single line - still don’t know how correct is this:

--- /usr/lib/python2.4/site-packages/twill-0.9b1-py2.4.egg/twill/wsgi_intercept.py.orig 2007-06-04 21:10:37 +0300
+++ /usr/lib/python2.4/site-packages/twill-0.9b1-py2.4.egg/twill/wsgi_intercept.py      2007-06-04 21:10:45 +0300
@@ -265,7 +265,7 @@
                 for data in self.write_results:
                     self.output.write(data)

-            if generator_data:
+            if generator_data is not None:
                 self.output.write(generator_data)
                 for data in self.result:
                     self.output.write(data)

Nevertheless it didn’t break anything and I have a luxury to test rss feeds too:

# -*- coding: utf-8 -*-
# $Id: tests.py 326 2007-06-04 15:58:48Z akhavr $

from nose.tools import *
from twill.commands import *
from twill import add_wsgi_intercept

def setup():
    import os
    os.environ["DJANGO_SETTINGS_MODULE"] = "web.settings"

    from django.core.servers.basehttp import AdminMediaHandler
    from django.core.handlers.wsgi import WSGIHandler

    app = AdminMediaHandler(WSGIHandler())
    add_wsgi_intercept("127.0.0.1", 9876, lambda: app)

@with_setup(setup)
def test_rss():
    'test rss feed'
    go('http://127.0.0.1:9876/kds-djangofeed/feed/rss/')
    code('200')
    find('<rss version="2.0"><channel><title>Django Feed Planet.</title>')
    return

Bookmark to:

Add 'testing django with twill and nosetests' to Del.icio.us

Add 'testing django with twill and nosetests' to digg

Add 'testing django with twill and nosetests' to FURL

Add 'testing django with twill and nosetests' to reddit

Add 'testing django with twill and nosetests' to Technorati

Add 'testing django with twill and nosetests' to Google Bookmarks

Add 'testing django with twill and nosetests' to Squidoo

Add 'testing django with twill and nosetests' to Bloglines

14 Responses to “testing django with twill and nosetests”

on 02 Dec 2007 at 8:50 pm akaihola

This works fine as you describe, except that for some reason I get the page content twice for every page.

I made a test view which only returns HttpResponse(’TEST’), and executing go(’http://127.0.0.1:9876/test/’) ; show() prints out “TESTTEST”.

Also showforms() for a simple one-form page displays the form twice.

I tried bypassing twill’s own browser and using twill._browser.MechanizeBrowser — same result.

What are the exact versions of Django and twill you used? I could try them first and, if they work ok, go on upgrading both gradually to newer versions to see if the problem appears at some point.
on 02 Dec 2007 at 8:54 pm Andrey Khavryuchenko

akaihola,

I’m using quite recently svn trunk of django.

I’ve not seen the behaviour you describe, so can’t help right now.

Does ’show’ itself dumps the content twice? Or something before it?

Try debugging the issue..
on 02 Dec 2007 at 9:19 pm Andrey Khavryuchenko

My twill is twill-0.9b1-py2.4.egg
on 02 Dec 2007 at 9:49 pm akaihola
I have Django newforms-admin branch rev 6710 and twill 0.9b1.

Something behind show() is doubling the content. As I said, I tried connecting directly with mechanize:
```
>>> from twill import _browser
>>> b = _browser.MechanizeBrowser()
>>> b.open('http://127.0.0.1:8000/test/').get_data()
closing...
closing...
'TESTTEST'
```
It\’s interesting that the \”closing…\” message is printed twice. Going one step deeper, I got the same result by calling b.mechopen(url) instead of b.open(url).

Continuing the debug session…
on 02 Dec 2007 at 9:56 pm Andrey Khavryuchenko

akaihola, please share the results of your debug session. I’ve not observed such behaviour and wonder what should cause it. On the other hand, I don’t check the output manually, I use regexp to match predefined criteria, so I might easily miss that.

I’m using django trunk (not newforms-admin) and now have r6702 - I update only every other week or so.

Looks like it’s an issue either with twill’s mechanize or django itself.

on 02 Dec 2007 at 11:58 pm akaihola

I set debuglevel to 2 (verbose) in twill.wsgi_intercept.py, and this is what it spits out:

connect: 127.0.0.1, 8000
INTERCEPTING call to 127.0.0.1:8000
>>> GET /test/ HTTP/1.1
Accept-Encoding: identity
Host: 127.0.0.1:8000
Connection: close
Accept: text/html; */*

>>>
HEADER: Accept-Encoding identity
HEADER: Host 127.0.0.1:8000
HEADER: Connection close
HEADER: Accept text/html; */*
METHOD LINE: GET /test/ HTTP/1.1

method: GET; script_name: ; path_info: /test/; query_string:
WSGI environ dictionary: {'wsgi.multithread': 0, 'SCRIPT_NAME': '', 'wsgi.input': , 'REQUEST_METHOD': 'GET', 'HTTP_HOST': '127.0.0.1:8000', 'PATH_INFO': '/test/', 'SERVER_PROTOCOL': 'HTTP/1.1\r\n', 'QUERY_STRING': '', 'wsgi.version': (1, 0), 'HTTP_ACCEPT': 'text/html; */*', 'HTTP_CONNECTION': 'close', 'SERVER_NAME': '127.0.0.1', 'REMOTE_ADDR': '127.0.0.1', 'wsgi.run_once': 0, 'wsgi.errors': , 'wsgi.multiprocess': 0, 'wsgi.url_scheme': 'http', 'SERVER_PORT': '8000', 'HTTP_ACCEPT_ENCODING': 'identity'}
closing...
*** HTTP/1.0 200 OK
Content-Type: text/html; charset=utf-8

TESTTEST ***

Getting close…

on 02 Dec 2007 at 11:59 pm akaihola

(sorry, I didn’t know how to mark up code, there’s no syntax reference link here)
on 03 Dec 2007 at 12:05 am Andrey Khavryuchenko

akaihola, I’ve fixed the formatting. Prefix each code line with 4 spaces or single tab - and it will be formatted like code.

Re log itself: obviously, the django method is called twice for some reason.
on 03 Dec 2007 at 12:28 am akaihola
Ok, I see where this is happening. The
```
twill.wsgi_intercept.wsgi_fake_socket.makefile()
```
method first writes page headers and payload from generator_data, then from self.result.

I\\’m not familiar enough with WSGI to be able to analyze whether it\\’s Django or twill which is behaving incorrectly.
on 03 Dec 2007 at 12:34 am akaihola

Here are some pointers to the relevant code on both sides:

In twill, see http://darcs.idyll.org/~t/projects/twill/twill/wsgi_intercept.py
around line 260 (their trac site seems to be down).

In Django, see the call method at http://code.djangoproject.com/browser/django/trunk/django/core/handlers/wsgi.py#L186

The startresponse() function from line 225 of wsgiintercept.py ends up as the start_response argument of WSGIHandler.call().

on 04 Dec 2007 at 12:34 am akaihola

Here is a super-simple test case. Put the files twilltest.py and urls.py in a directory, make sure Django and twill are in PYTHONPATH and run the test by typing “PYTHONPATH=. python twilltest.py” in the directory.

twilltest.py:

#!/usr/bin/python

from django.conf import settings
settings.configure(ROOT_URLCONF='urls')

from django.core.servers.basehttp import AdminMediaHandler
from django.core.handlers.wsgi import WSGIHandler
app = AdminMediaHandler(WSGIHandler())

from twill import add_wsgi_intercept
add_wsgi_intercept('127.0.0.1', 8000, lambda: app)

from twill.commands import *
go('http://127.0.0.1:8000/test/')
show()

urls.py:

from django.conf.urls.defaults import patterns
from django.http import HttpResponse

urlpatterns = patterns('',
    (r'^test/', lambda request: HttpResponse('TEST')),)

Output on my Linux box with both Python 2.4 and 2.5 and twill 0.9b1:

closing...
==&gt; at http://127.0.0.1:8000/test/
TESTTEST

on 04 Dec 2007 at 12:44 am akaihola

I asked for help on the django-developers newsgroup:

http://groups.google.com/group/django-developers/t/c02ebeb8e9910953

The twill site seems to be down right now, so I’ll have to approach them later.

I did try my test case with today’s up-to-date checkout of both Django trunk and newforms-admin. Both double the page content.
on 04 Dec 2007 at 1:41 am akaihola

Result of further investigation:

twill 0.8.4 worked correctly, 0.8.5 introduced this duplicating behavior.
on 04 Dec 2007 at 11:50 pm akaihola

Good news: Titus promptly tracked down a bug in twill, and this issue is now fixed in twill-latest.

The package is available at http://darcs.idyll.org/~t/projects/twill-latest.tar.gz

Download the tarball and run:
easy_install twill-latest.tar.gz
(if you have setuptools installed)

You must be logged in to post a comment.

testing django with twill and nosetests

14 Responses to “testing django with twill and nosetests”

Leave a Reply

Search

News