Difference between revisions of "SoC test framework revisited"

Revision as of 16:31, 13 February 2011

Introduction

After laying the groundwork in the 2009 GSoC project, it is now time to take unit testing in Geeklog one step further. We currently have some tests set up to run automatically after every checkin to the Geeklog Mercurial repository, but those only cover a very small portion of the code base.

Note: In this project, we are not interested in doing any GUI testing (yet).

Incentive

The main goal of this follow-up project is to bring those parts of Geeklog under test that require a working database and integrate those tests into our continuous integration setup.

A mock XML database was developed as part of the previous project. In this follow-up project, we would like this decision to be revisited. Is this the best way to go forward? If so, how can we prevent it getting out of sync with database changes in Geeklog? If it isn't the best option (e.g. missing write access is a problem that would need to be solved) then what are the alternatives?

Once this basic decision has been made, the main goals would be to

• bring lib-common.php under test
• bring at least one plugin under test

Details

One of the central pieces of Geeklog is the lib-common.php file. It contains a collection of commonly used functions (hence the name). When loaded, lib-common.php will also create a session for the current user (and do all the related work, like loading preferences, setting up permissions) and load the installed plugins. It also pulls in many additional libraries and classes. This file can not be loaded without a properly set up database, which makes it hard to put under test.

Things to look into:

• Do we go with the mock XML database or do we switch to something else (see above)?
• Review the directory layout. The current layout was chosen in part to be able to run a GUI for the unit tests. With CI servers such as Jenkins being available, is this still necessary? If not, should the directory layout be changed (simplified)?

Goals:

• Bring lib-common.php under tests, i.e. write tests for at least some of the COM_ functions from that library (which is not possible without solving the database problem first).
• Demonstrate how to write tests for a plugin. By bringing one of the bundled plugins under test, authors of 3rd-party plugins will be able to set up their own test environments and write tests for their plugins.

Level of Difficulty

medium

A word of warning: This project has the potential to cause some frustration initially while trying to bootstrap things.

Further Reading

• Description of the current Test Suite
• Description of the 2009 GSoC Test Framework project
• Mercurial repository of the Test Framework