Department of Computing

Local Navigation

COMP249: Web Development Assignment

Updates

In this assignment you will implement a simple website that allows users to store bookmarks and share them with others. An example of this kind of service can be found at del.icio.us which is an inernet startup that was aquired by Yahoo last year. Your project will not be as complex as del.icio.us but will offer some of the core functionality. This problem was chosen as it illustrates all of the basic components of a user facing web service.

Completing this assignment involves a number of distinct steps:

  1. Allow users to register with your service, store their details in a database.
  2. Allow users to log on to the service, provide customised pages for each user based on thier stored data.
  3. Allow users to store a bookmark with associated data, store this in a database.
  4. Generate pages from the bookmark store for individual users and for everyone.

An example of the running application can be found at: http://www.ics.mq.edu.au/~cassidy/cgi-bin/main.py. This is intended as a guide only, your application doesn't need to look exactly like my version. My version might also be missing some of the requirements in this document.

You can download a starter kit for this assignment here: comp249app.zip. The kit contains the source code described below as a starting point for your own development. The code in the kit will run but will only generate a main page of the application (and the main page isn't correct).

This assignment is due in two parts. The first submission consists of some Python code in a single file. The final submission is for the entire working application. The due dates and requrements are set out at the end of this page.

Database

We will use the SQLite database to store application data in this assignment. SQLite is a fully featured SQL database system which can be easily embedded into applications. It does not require a large scale server installation like most other relational database systems. Databases are stored in files which can be copied between systems very easily. SQLite implements most of the SQL standard and is an excellent development platform for web applications. If a larger scale database is needed for deployment, the same SQL code will usually work without modification. SQLite is commonly used as an embedded data store, for example, many Apple products use SQLite to store data.

SQLite provides application interfaces for many programming languages, we will use a Python interface called pysqlite which implements Python's standard database API for SQLite.

A single database with multiple tables will be used. The table schema are defined as follows, with some commentary on the use of each field. You may not modify any of the tables in the database schema, however, you may add new tables if you wish.

CREATE TABLE users (
    email VARCHAR NOT NULL PRIMARY KEY,  
    fullname VARCHAR,
    country VARCHAR,
    homepage VARCHAR,
    password VARCHAR
);
CREATE TABLE bookmarks (
    id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
    user VARCHAR, 
    url VARCHAR, 
    title VARCHAR,  
    date VARCHAR DEFAULT CURRENT_TIMESTAMP
);
CREATE TABLE tags (
    id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
    bookmark INTEGER,
    tag VARCHAR
)
CREATE TABLE sessions ( 
    sessionid VARCHAR NOT NULL PRIMARY KEY,
    user VARCHAR
);

Note that the id fields in bookmarks and tags tables are AUTOINCREMENT fields meaning that their values are automatically set when a new row is entered. Also, the date field in bookmarks will automatically default to the current date and time if a value is not provided. See the example code in comp249db.py for examples of how to use these features.

The email field in the users table is intended to be the same as the user field in the bookmarks and sessions tables: that is, it is the unique identifier for users in our system. More details on how these tables are used is given below.

Passwords

Storing plain text passwords is a security risk (if someone gains access to the data store they can compromise all user accounts) so it is common to store an encrypted version of the password. The python sha module provides a one way hash function which serves well as an encrypted encoding of a password. The following function can be used to derive the encrypted form of a password: 

def pwcrypt(string):
    """Returns the crypted version of string for password
    storage and checking"""
    return sha.new(string).hexdigest()

This function is included in the comp249db.py file described below.

Database Code

You are provided with a Python module comp249db.py which provides a simple interface to the database, initialises the database and creates some initial data. The intial data includes the following:

The comp249db.py module provides a connect() method which should be used to connect to the database. It ensures that the same database location is used each time. In testing your code, I will change the database location so you should not include any reference to it in your code. You can set the database location at the start of the comp249db.py file, do not change anything else in this file.

As an example of using the comp249db.py module, the following code lists the current bookmarks table:

import comp249db

conn = comp249db.connect()
cur = conn.cursor()
cur.execute("SELECT * FROM bookmarks")
for row in cur.fetchall():
   print row

The file comp249db.py contains initialisation code at the end so that when it is run, the database will be created and filled with sample data. You must initialise the database before you start working on the application. To do this, just double click the file in Windows (as long as Python is installed) or run it from IDLE. Take care to edit the directory at the top of the file before you do this so that you know where the database will be stored.

Code Provided

In addition to the database interface, you are provided with a basic implementation of the main.py page that lists the current entries in the database, and a file page.py that implements a simple page generation system.

main.py

The structure of main.py is designed to allow both automated testing, importing into other modules and running as a CGI script. The module defines a procedure handle(form) which takes a cgi.FieldStorage() object and returns the full response that the CGI script should make to that form. At the bottom of the file is the usual Python trick for code that should run if this module is being executed directly - eg. as a CGI script. Here we create a FieldStorage object and print the result of calling handle. On the other hand, if we are importing this module into another module, we can easily generate the main page output by calling main.handle(form) - this allows any other module (eg. the login.py module) to generate the main page if it needs to.

You must retain this structure for main.py since we will rely on it for automatic testing. You are strongly advised to use the same structure for your other pages.

page.py

This module implements a simple HTML generation scheme using templates. The procedure page.render takes a dictionary object containing values for the keys title, content and id and generates HTML output (as a string) by substituting these values into the content of a template file. Substitution is done using the built in Python % operator. Templates are found in the templates directory of the application. I have provided one template file as an example, main.html which you can modify and copy to create the different pages needed by your application. The use of page.render is illustrated in main.py:

    info = {
      'title': "myUnits Main Page", 
      'content': list_units(form), 
      'id': 'main'
    }
	
    return page.render("main", info)

Here the main.html template will be used, %(title)s will be replaced by myUnits Main Page and %(content)s will be replaced by the output of the list_units procedure. The procedure as implemented will replace any key in the template, so if you want to introduce a %(user)s key for example, that would be replaced with the value of the key user in the dictionary.

CGI Server

Also included in the download is a simple Python CGI web server. I'd recommend using this as part of your development environment rather than installing a larger server or working on a remote host like Platypus. In testing your code I will use a similar server so there's no requirement in this assignment that your code runs on Platypus. To run the server on Windows with Python installed, just double click on cgiserver.py, you should see the window pop up which invites you to connect to http://localhost:8000/ where you should be able to connect to your application.

Requirements

Your task is to complete the bookmarker application so that it provides a useable service and looks and works like a professional, modern web application. This will involve writing additional Python code, HTML templates and CSS stylsheets. Your finished application should have at least the following different pages:

The pages should be generated by the python files listed above (as illustrated by main.py) but note that some files need to both generate an input form and process the response.

In addition, you must implement a small number of Python procedures in the file util.py, these will be automatically tested. The procedures to be defined are listed in the util.py file supplied. They are briefly:

A test script test-util.py is provided to test the procedures in util.py using a unit testing framework. Running this script from the command line or via IDLE should show the result of the tests. If there are no errors you should see the following output:

Test that util.store_bookmark() correctly stores a bookmark ... ok
Test get_bookmarks returns ids of stored bookmarks in order ... ok
Test that util.check_login() verifies a stored password ... ok
Test that the logout function removes the session entry ... ok
Test that the util.register_user() call adds someone to the database ... ok

----------------------------------------------------------------------
Ran 5 tests in 0.167s

OK

If there are errors, you should see a message that gives some information about what went wrong. Read this carefully to try to diagnose the problem. For example, if the login test fails you might see:

...
Test that the logout function removes the session entry ... FAIL 

======================================================================
FAIL: Test that util.check_login() verifies a stored password
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test-util.py", line 98, in test_login
    self.failIf(list==None, "Can't find session in database")
AssertionError: Can't find session in database

Here, the test script can't see the session in the database after the check_login function is called.

User Sessions

To use the application, a user must first register or login; following successful registration or login, a cookie will be returned to the user with the HTML response. This will be a session cookie, set to expire when the user closes the web browser. This cookie will have the name session and the value will be a random string. The random string (session key) will be stored in the sessions table in the database alongside the user's email address. While an entry for the session key is in the database your applicaton can identify the user via the cookie that will be returned to you. If you can identify the user in this way, you will serve up customised versions of pages (ie. the main page lists this user's ratings). It also makes sense not to display the registration link if a user is logged in and not to display the rating link if there is no user logged in.

You should also provide a logout option which removes the current user from the sessions table thus invalidating the cookie which they hold. This is implemented via the util.logout procedure.

How to Get Started

This is potentially a large assignment but if you go about things the right way, it should be manageable in the time given. Firstly, don't be put off by the scale of the project; the details above is there to help you and make sure that we can mark your work in reasonable time. Here's a suggested work plan:

  1. Work on the functions in util.py, to do this you don't need to think about the web interface at all, just get them to return the right values and modify the database appropriately. Within this file, register_user and check_login are good starting points.
  2. Use the get_bookmarks function to rewrite the main page according to the specifications.
  3. Write the registration and login pages, here you need to deal with cookies properly.
  4. Now generate the user specific main page, a modification of what you did in the earlier. Make this page appear after login.
  5. Deal with storing new bookmarks.

Assessment

This assignment is worth 15% of the final marks for COMP249. The work will be assessed in two parts: in the first submission the procedures in util.py will be automatically tested; the second submission will assess the entire application.

The first submission is worth 5 marks and will check that the procedures in util.py conform to their requirements using an automated test script. An example of this script is provided in the inital starter kit. These marks will be given purely on the basis of performance against the test script. If you do not make the first submission, you will not be able to make up the marks in the final submission, even if your util.py code works perfectly.

The final submission should consist of a complete working implementation of the application as a set of Python scripts, stylesheets and page templates. This will inculde util.py as submitted earlier. Your work will be assessed by semi-automated testing (this is why we need to be able to register and login via the util.py procedures), by viewing your site and reading your code. You should make full use of in-file documentation in your Python code to provide a commentary on your design and implementation; for example, use the initial documentation string in each file to describe what is being done and how it is implemented. You might use the main.py file to hold overall documentation of your application.

Marking criteria for the final submission are:

Your final grade will be calculated based on your performance against the above criteria. It is not necessary to get a distinction against all criteria to get full marks for the assignment.

Submission

You should submit your assignment via WebCT. Please be aware that there have been issues with files not being uploaded on WebCT. Please check that your submission is there and contact me if you think there might be a problem.

Part one of the assignment (util.py) should be submitted before 5pm, Friday 28th April but you may submit before that date. It will be marked and returned as soon as possible after submission.

Final submission is due on Monday 5th May at 5pm. You should submit all of the files needed to run your application as a zip file using the same directory structure as the provided code (one directory containing cgi-bin and templates subdirectories). You should include any images or auxiliary files needed by your application.

Please remember that comp249db.py shouldn't be modified except for the location of the database at the top of the file and that you should only connect to the database using comp249db.connect(). Forgetting this will mean that you will almost certainly fail the automated tests and that your application won't work when I try to run it. Similarly, please don't modify the database tables since my testing code will be looking at their contents.

Submit your assignment via the submission system on WebCT.

Comments to Steve Cassidy

Copyright & Site information

  • CRICOS Provider No 00002J, ABN 90 952 801 237
  • Authorised by: HOD