A few days ago I uploaded a webbased GUI for PHPUnit from Sebastian Bergmann to sourceforge. as part of the Cool project. I use it myself very intensively and it could be very useful for others as well. But oh, the download statistic is pretty bad. Time to blog about it and some other pulsating evolutions.

The cool address is http://cool.sf.net.

PHPUnit is the standard for unit testing with PHP. It runs pretty well on command line as well as in continous integration environments. The documentation from Sebastian Bergman is excellent and freely available. But pretty GUIs for it are still rare.

The address of PHPUnit: http://phpunit.de

Robert Lemke did a first approach to bring up a GUI as a T3 extension, with some interesting concepts. But it was still a little difficult to work with it. Now Kasper Ligaard from Denmark did a big job to improve Roberts extension and the result is very promising.

The extension key is phpunit.

Daniel Brüßler from Germany assisted by improving the PEAR installer extension, so that you can easily update PHPUnit via PEAR channel, as often as a new version has been released.

The extension key is PEAR.

Mario Rimann and Oliver Klee stepped ahead an improved the quality of the Seminar Manager extension by unit testing.

I hope we will see more well tested TYPO3 extensions in future.

To have a nice GUI available also outside the world of TYPO3 I migrated the Ideas of Robert and Kasper to PHPUnit_CoolUI. Another source inspiration is
PHPUnit2 HTML Runner from Dave Spurr (UK). Unfortunately it depends on an older version of PHPUnit, so that I didn’t get it running.

DRY,

Don’t Repeat Yourself (DRY, also known as Once and Only Once or Single Point of Truth (SPOT)) is a process philosophy aimed at reducing duplication, particularly in computing. The philosophy emphasizes that information should not be duplicated, because duplication increases the difficulty of change, may decrease clarity, and leads to opportunities for inconsistency. DRY is a core principle of Andy Hunt and Dave Thomas’s book The Pragmatic Programmer. [0]

[0] Wikipedia: http://en.wikipedia.org/wiki/Don%27t_repeat_yourself/

Before searching for a way to, apply Phing [1] to T3 extensions I tested it in a small independent project (PHPUnit_CoolUI [2]), that I could configure as I pleased. And it was indeed a pleasure. After having learned and integrated Doxygen [3], Docutils [4] and Phing I was able to process a full new release with one click. Editing, updating the source files, INSTALL, README, TODO, FeatureRequests, ChangeLog, etc. with version and path informations. Including a lot of this informations automatically into the manual and finally packing it all to a tarball for release.

DRY, this self-evident principle is not always that simple to meet and far away from beeing systematically applied within TYPO3. Just the opposit. It’s hard to publish a clean extension, that respects all coding guidelines, because you have to place the same information into different places repeatedly for each release.

Take the changelog as example. There is a ChangeLog file in the extension. The changelog information should be appended to the manual and also pasted into the upload form to TER to be displayed on typo3.org. It’s similar for the TODO. The current version number should ideally be added to all class files, that have been altered and nearly in every involved file and document.

Well, there are solutions. CVS [5] and SVN [6] can automatically update version markers in the sources. API generator tools can apply version informations to the generated API, that is configured in the configuration. Document generators offer markup for automatic insertion of informations like version strings. All this can simplify the maintainance of software, but you still have to manually maintain the configuration files of the different tools to set the current properties.

Another step is needed, to autoupdate the differnt configuration files and to run and controll the generators and tools efficiently. Nothing special, no need to reinvent the wheel. Programming of compiled languages like C or Java can’t be done efficiantly without a tool for this kind of stuff. There are different Makes [7] for C, C++ there is Ant [8] for Java. Certainly you could write simple shell scripts to reach similar targets or use a scripting language like PHP.

PHP doesn’t need to be compiled and the project management automatisms can be coded in PHP themself. Reasons why tools like Ant havn’t been available for PHP early. However, since a few years Phing is availble implementing conceptions of Ant for PHP.

Setting up a development environment with Phing may sound oversized for a little freelancer developing a small TYPO3 extension. In the moment you want to maintain a public open source product over multiple versions, with user and admin documentation, with licences, maybe with a website, such an environment becomes extremly handy. Not to speak of a bigger project team
with multiple persons.

What system do you use?

Should we set up such an environment for extension development for example by enhancement of the extdev extension?

What tools would you recommend to include?

[1] Phing: http://phing.info/docs/guide/current/
[2] PHPUnit_CoolUI: http://cool.sf.net
[3] Doygen: http://www.stack.nl/~dimitri/doxygen/
[4] Docutils: http://docutils.sourceforge.net/
[5] CVS: http://www.nongnu.org/cvs/
[6] SVN: http://subversion.tigris.org/
[7] Make: http://www.gnu.org/software/make/manual/html_node/index.html
[8] Ant: http://ant.apache.org/manual/index.html

Today I upgrade from Ubuntu 7.04 (Feisty Fawn) to the release candidate of Ubuntu 7.10 (Gutsy Gibbon). During the update the logical volume /usr comes to it’s limits. I need to add 1 GB of diskspace. It’s a reiserfs. I should be able to extend the running mounted volume.

The overall steps are 3

1. Backup first
2. Extending the logical volume
3. Extending the filesystem on the volume

Extending the logical volume

Show the current state of the logical volums.

lvdisplay

To extend the logical volume by one GB is simple. lvm help leads to the required parameters.

lvresize -L +1G /dev/uvg/usr

With the -t Parameter I can run a test before.

lvresize -t -L +1G /dev/uvg/usr

Extending the filesystem on the volume

To resize the reiserfs to the full new size of the logical volume is still more simple.

resize_reiserfs /dev/uvg/usr

Displaying the success.

lvdisplay

Troubleshouting

After extending the logical volume I can’t run the command resize_reiserfs as expected because the ‘/dev/uvg/usr’ is not found any more. The moment of panic. Good if you have a backup. But I don’t need it. A little search in the Web brings me to a solution.

dmsetup mknodes

This lowlevel command rebuilds the lost mappings of pathes and devices and everything runs fine again.

yesterday I had the first opportunity to test the expandable controller
feature of lib. To remember: It’s a way to register an additional
collection of action methods to an existing controller class, either from
within the same extension or from a foreign one.

My current extension has a single plugin controller class with already many,
many actions. No I needed to extend the functionality by another table and
form. Because the main controller is already very large I decided to
encapsulate the new set of actions into a separate class. First praxis test
of the expandable controller.

It didn’t run out of the box and I started to debug. It turned out, that
additional controller class needs to be included before calling it by the
main controller. Well, that may sound logical, but I didn’t. There was no
error message, the registered controller simply was ignored. That made
debugging difficult. The smart solution would be to autoinclude the
registered controllers in the main controller. I need to file a bug report.

I included the additional controller in ext_localconf.php. Now everything is
running well. It really seems to be a very smart solution, comparing
projects in the past, where a needed to overwrite large switch-case
constructs in multiple layers, each time for every extension of the
extension.

1. Clear the caches.
2. Check the rights.
3. Check the pathes.
4. Include the static template.
5. Open your eyes.
6. Oops, it did work already …

Everybody is speaking about Model-View-Controller architecture (MVC). Has TYPO3 a MVC architecture? What do you think?

Many developers will answer this question with a clear YES. The data are in a database. Obviously there must be a model. TYPO3 generates webpages from this using HTML templates. So there is a view. Bingo. But stop. First questions pop up now:

• Having a database layer, an application-server layer and a client layer (browser) makes it a three-tier architecture. But that is not the definition of MVC.
• What exactly is the controller?
• Why do we speak of TS templates?
• Does typoscript belong to the view or is it rather the controller?

Let’s think twice. There is the file index.php, setting up the environment, reading the configuration, loading libraries, connecting the database, finally starting the rendering, caching and output of the TS tree. We can call this script the outer TYPO3 framework without counting it into MVC. If we want to fit it into MVC we should probably count it to the controller.

Having a controller now, we could call the database the model and the whole TS tree the view. The name TS template matches. Originally there were no HTML templates. All HTML was generated by TS templates. Perfect. We could take it this way.

But let’s think thrice. Let’s take a closer look into the TS tree. There are objects receiving request parameters. Typically that is a job of the controller. There are objects, that query the database. Typically that is part part of the model.

Taking TS objects into the focus we can’t state any longer, that the TS tree is the view. It’s rather a mixture of model, view and controller itself. The objects are nested. The page contains the HTML template. The template is filled with columns. The columns contain content objects. The content objects can contain extensions and each content object or extension can have functionality of model view and controller.

What holds this tree together? Most objects deliver a result string of HTML to their parent objects, strings that are finally composed to the HTML page. We discover an architecture that is hierarchically splitted into many MVC nodes, that recursively build the common nested output.

We can’t really say that each of the nodes has a clean separation of model, view and controller, but many nodes have a MVC separation. A tree of rather independent MVC nodes, that build a nested view is the definition of the Presentation-abstraction-control architecture.

All in all we can point out two different dimensions of flexibilty in TYPO3. First there is the flexibiltiy of the MVC separation. Second there is the flexibilty of nested objects configured by TS. Breaking the MVC pattern down to such a finegrained level has a price. To exchange the whole view, means to exchange it in every little node separetly. But that is a theoretical case.

The more finegrained the tree, the less advantages are in the MVC design. The more finegrained the tree, the more economic it becomes to exchange whole tree branches, instead of multiple model, view or controler parts. In practice you will exchange a whole grapical menu against a text menu, even if the model and view parts work identical and you could go by exchanging the view parts. This is probably the reason why the MVC pattern isn’t available in each leaf of the TS tree, the reason that we find some leafs of a monolithik, non MVC design, in example in form of many extensions.

Comming back to the original question, we now say that TYPO3 has not a MVC architecture of three parts. It is a tree of many nodes which can have a MVC design each. We discover that TYPO3 has mainly a PAC architecture.

Subline: TYPO3 has a mixture of different architectures. “Macro” MVC design for the HTML template. “nested micro” MVC (PAC) for almoust all TS nodes and a monolithic design in few nodes. Each architecture is used, where it matches best – or it simply depends on the taste of the developer of the given node. This mixture of architectures is one reason why TYPO3 is that mighty. It’s also one reason, why TYPO3 is that complex to understand.

Typolink matters

Links are the magic stuff that keep the web together.
It would be understatement to say that links characterize the web.
They simply make the web. No links, no web.

Links get even more important for a content management system like
TYPO3. They not only point from one page to the other, they even controll
the whole system with all the many parameters. The parameters can
be visible for the user or hidden behind hash values or “pretty URLs”.

If this parameters are not perfectly done, the system starts to
fail. Language settings get lost, frames jump out of a frameset,
caching doesn’t work, the page is getting slow, XHTML becomes invalid,
JavaScript breaks, and and and …

You couldn’t write all this parameters in all this links by hand.
Even if you could, you would need to rewrite them all if an additional
parameter is required. For the same reasons you can’t simply patch up
a link in a PHP script. You have to use the one and only function, that
automatically mananges all the parameters without flaws. This function
is the famous “typolink” function for TYPO3.

This function is called indirectly by TS and some other functions, but
for sure if you program PHP you should be able to use this function directly.

Overview

The typolink is a function of the class tslib_cObj. The first parameter is
the linktext, the second an array with multiply configurations to controll
the links behaviour. This configurations are nicely documented in the
documentation TSref, where they are called properties.

So to use typolink, you perpare the controlling array, get an instance of
tslib_cObj and execute the function.

Example:

1.) Preparing the controlling array


$configurations['useCacheHash'] = 1; // make it a caching link
$configurations['parameter'] = 123; // target page id or external Url, Email, etc.
$configurations['section'] = 'par7'; // id for an anchor
$configurations['target'] = 'blank'; // set a frameset target for internal links
$configurations['extTarget'] = 'blank'; // set a frameset target for external links
$configurations['ATagParams'] = 'class="exampleClass" '; // set a class for the tag
$configurations['ATagParams'] .= 'id="exampleId" '; // set an id tor the tag
[ . . . ] // more attributs of the a tag
$configurations['additionalParams'] = 'myExtension[key1]=value1&myExtension[key2]=value2'; // Extension parameters


Hint: How to build proper additionalParams with Umlauts?

2.) Creating the object …

$cObject = t3lib_div::makeInstance('tslib_cObj');

Hint: Consider to use a singleton, if you create multiple links.

… or just use the current cObj of the plugin

$cObject = $this->cObj;

3.) Rendering the link tag or the plain link URL:

$linkTag = $cObject->typolink('Example text of the link', $configurations);

or


$configurations['returnLast'] = 'url'; // get it as URL
$url = htmlspecialchars($cObject->typolink(NULL, $configurations));


Hint: Typolink generates the link tag with htmlspecialchars. For the URL you have to do it yourself.

Doing it the object orientated way

Maybe you prefer to do it the object orientated way. You find a full working link object in the extension “lib”. You can use it directly or copy and rename it to make it part of your extension.

From 1.650.000 to 2.460.000 – disaster in progress

3 monthes passed. It’s time to do the test again.

April 17. 2007 — Juli 27. 2007
2.050.000 — 2.550.000 (google.de: “no_cache=1″)
1.650.000 — 2.460.000 (google.de: inurl:”no_cache=1″)

While in april there there were 20% fewer hits when prepending “inurl:” to the search this difference is 5% today. This indicates that the algorithm may have been modified. That should be considerd when looking at the results.

The results of both queries exploded, ca. 25% for the first search term, ca. 50% for the second.

Will the TYPO3 community be able to invert the trend to a growing amount of ill caching pages?

Some people say, “Don’t trust a developer that didn’t write his own compiler.”. A few days ago I finished the first version of my first compiler. It compiles typoscript to a php array.

Source and Binary for LInux

a.line.of = Typopscript

$TS['a.']['line.']['of'] = 'Typoscript';
 

It is implemented using Flex and Bison the open source implementations of Lex and Yacc. The result is a little C progam. You can feed TS into the compiler from within any PHP script and get a PHP array back in return.

Is that a compiler? Doesn’t a compiler build binaries not arrays?

Well, one definition is that a compiler is a translator. It typically tokenizes the input in one language into atomic pices, checks the syntax and maybe sematics and transforms it into another language. In traditional compilers the target language is machine code. In newer compilers it is often C. In this case it’s PHP and it’s focused upon compiling an array.

What is it good for?

If you have discovered the advantages of TS, it’s likely that you want to use TS not only within TYPO3. In the moment I want to use it for configuration within a non-TYPO3 PHP script.

Could it be used in TYPO3?

Certainly it would be possible to use it within TYPO3 itself, to get a faster TS engine. Speaking error messages, even checks of semantics could be implemented. That are features of the compiler approach.

For now it’s only a prototype and some features are still lacking to use it as a replacement for the PHP based TS engine. It will also be a bunch of work, to build a PHP extension around, that provides exaclty the same API of the current TS engine, including the handling of conditions etc. We will see if I find the time and drive to do such an implementation.

Introduction

Often people like to have a questionnaire with multipage forms, that they can setup and configure on their own. They would like to work with a simple configuration list of questions or with the assistance of a wizzard. It should be able to branch into subforms depending on the selected answers.

Motivation

I have different ideas, how this could be done and want to scratch down my ideas for the first model with a very abstract concept, that I call a pure YES-NO questionary. The other ideas are based on the YES-NO concept. Just like you can build complex programs and data on top of the primitive 0 1 bit structure.

Axiom

My basic assumption is, that you can break down every controlling question (not the text collecting ones) of a questionary to plain YES-NO questions. A checkbox is a YES-NO question. Radiobuttons are list of YES-NO questions, where only one question can result in YES. Multiselections are a list of YES-NO questions, where multiple questions can have a YES answer.

Coins

We bundle all questions, that are necessary to be asked, into a first form. For every question that is answered with a YES we can branch to a subform. The subform itself is assembled in the same way from YES-NO questions, that can branch to further subforms. The result is a tree of forms that would fully be passed, if all questions would be answered with YES. If all answers turn to NO, only the initial form is to be passed.

So the coins of this model are:

• form tree
• forms
• YES-NO questions
• YES-NO answers
• pointers form answers to (sub)forms

Implementation

How are the subforms called for each positive answers, while there are multiple questions within one form?

Multiple solutions are possible.

• A new window in example in form of a popup.
• Inline inclusion based on AJAX.
• Postprocessing after submit of the previous form controlled by a stack.

How does the stack work?

Every positive answer would be pushed on top of the stack. As soon as the form is submitted and checked, the stack would be processed. We pop the top entry and replace it by all positive answers of the matching subform. This is done recursively until all forms are done and the stack has been reduced to zero.

Logically not every positive answer has to result in a new subform else the tree would grow infinitely. Positive answers can be leafs of the tree. Then the information is just gathered with all the other results.

When is the collected data stored?

Two basical concepts are possible. Either we store the data directly after submit of each form. This solution is very stable and the vistor can come back at any time to proceed a previously started form fillout. The disadvantage is, that we collect entries into the DB from fillouts that are never finished. The alternative is to store all the data into the session and to store it only if the whole precess has been done. The disadvantage is, that all data will be lost, if the vistor pauses to long.

What happens, if the visitor steps back some forms, to change a YES to NO or vice versa?

If a previous YES is chanted to NO this cuts the complete subbranch. That can be done in several forms:

• The data of the subbranch is immediatly removed from the session or the database.
• The data is kept and removed upon the final submit of the whole transaction.
• The data is kept and stored. In this case it’s necassary to evalute the blocking NO value for the subbranch upon usage of the collected data.

For each NO, that is changed to a YES, the appropriate action needs to be executed, depending by an AJAX include or a popup or by pushing the stack. If we did a previous clearing of a branch during an alteration from YES to NO we need to cycle the branch again.

How is it all configured?

The coins are given. Each form contains a list of questions. Each form needs an ID for the answer form relation. Each question can contain a pointer to a form ID for the case of a positve answer. Thats it. WE can configure it in any configuration language i.e. in — surprise — TypoScript.

How is it controlled?

I suggest an implementation based on the lib/div controller.

What about a wizzard?

If we extend this basical concept a little and use the form pointer in form of a loop and add some other inputs to pick up the questions, a wizzard can be implemented with the YES-NO questionary itself.