Toward the Design Quality Evaluation of Object-Oriented Software Systems

Fernando Brito e Abreu (INESC/ISEG)Miguel Goulão, Rita Esteves (INESC/IST)

INESC, Rua Alves Redol, 9, Apartado 13069, 1000 Lisboa, PORTUGAL(phone: +351-1-3100226 / fax: +351-1-525843 / email: fba@inesc.pt)

ABSTRACT

This paper presents some advances towards the quantitative evaluation of design attributes of object-oriented softwaresystems. We believe that these attributes can express the quality of internal structure, thus being strongly correlatedwith quality characteristics like analyzability, changeability, stability and testabilility, which are important tosoftware developers and maintainers.

An OO design metrics set is reviewed, along with its rationale. An experiment for collection and analysis of thosemetrics is described and several suppositions regarding the design are evaluated. A considerable number of classtaxonomies written in the C++ language were used as a sample. A tool to collect those metrics was built and used forthat purpose. Statistical analysis was performed to evaluate the collected data. Results show that some designheuristics can be derived and used to help guide the design process. It was also clear that a number of follow-uptopics deserve further research.1. INTRODUCTION

The backbone of any software system is its design. It is the skeleton where the flesh (code) will be supported. Adefective skeleton will not allow harmonious growth and will not easily accommodate change without amputations orcumbersome prothesis with all kinds of side effects. Because requirements analysis is most times incomplete, wemust be able to build software designs which are easily understandable, alterable, testable and preferably stable (withsmall propagation of modifications). The Object-Oriented (OO) paradigm includes a set of mechanisms1 such asinheritance, encapsulation, polymorphism and message-passing that are believed to allow the construction of designswhere those features are enforced. However, a designer must be able to use those mechanisms in a “convenient” way.Long before the OO languages became widespread, it was possible to build software with an OO “flavor”, usingconventional 3rd generation languages. Conversely, by simply using an OO language that supports those mechanismswe are not automatically favored with an increase in software quality and development productivity, because itseffective use relies on the designer’s ability. Being a “creative” activity, where multiple alternatives are oftenavailable for the same partition of the system being modeled, design would greatly benefit if some heuristics couldhelp choose the way. Design metrics are being used for this purpose.

Several research works in the OO design metrics arena were produced in recent years [Dumke95, Sellers95,Campanai94, Cant94, Chidamber94, Hopkins94, Abreu93]. However, there is a lack of experimental validation.Worse than that, there is scarce information on how the proposed metrics should be used. Facing the available metricsliterature, software practitioners are often left with the unpleasant feeling that “not everything that counts can becounted, and not everything that can be counted counts”2. A better scenario can be found on the field of OO reusemetrics, where experimental studies like [Melo95, Lewis91] are shedding some light.

An earlier paper [Abreu94] proposed the MOOD3 set of metrics. These metrics allow the use of the main mechanismsof the Object-Oriented paradigm to be evaluated and are reviewed here. They are supposed to help establishcomparisons and derive conclusions among heterogeneous systems (different size, complexity, application domainand/or OO implementation language), thus allowing cumulative knowledge to be achieved. Although the languageheterogeneity is not yet addressed in this paper, an experiment is described where the sample (OO systems fromwhich the MOOD metrics were collected) is a good representation of all the other differences.

This paper is organized as follows: the next section introduces the main goals and strategy of the current researchwork from which this paper originated. Section 3 includes the detailed review of the MOOD set along with itsrationale. A simple case study in C++ is used to illustrate the basic concepts. The following section describes anexperiment of systematic collection of the MOOD metrics, including the tool used, the target sample and the resultsachieved. Section 5 discusses the experimental results and proposes some design heuristics based on the MOOD set. 1 - Some of those are a natural evolution of concepts and constructs present in structured programming and founded on abstract data type theory.2 - Albert Einstein

3 - Metrics for Object Oriented Design

Some new research directions emerging from this study are mentioned in section 6. The final section presents aretrospective overview.

2. RESEARCH GOALS AND STRATEGY2.1 Main goals

The research being carried out in this area by the Software Engineering Group at INESC4 in cooperation with theLisbon Technical University is centered around two main goals:

Goal 1 - Improve the OO design process to achieve better maintainability

Maintenance is (and will surely continue to be) the major resource waster in the whole software life-cycle. Main-tainability can be evaluated through several quality sub-characteristics like analyzability, changeability, stability andtestabilility [ISO9126]. Object-orientation is believed to reduce the referred effort waste, if its basic mechanisms areused conveniently. We believe and expect to prove that, at the system level, there are patterns for the extent of use ofencapsulation, inheritance, polymorphism or cooperation among classes which are closely correlated with thosequality characteristics. By finding those patterns, through thorough experimental validation, we do not expect toidentify a good design when we see one, but rather to say that a certain design is more maintainable than another.This is particularly useful for inexperienced designers, often faced with a combinatorial explosion of arbitrary designdecisions.

Goal 2 - Improve the OO estimation process to achieve better resource allocation

Producing effort and schedule estimates for OO software development requires evaluating the size and complexity ofthe system to be built. The percentage that is going to be built from scratch and the percent that is going to be reusedwith minor or major adaptations (from existing component libraries) must also be evaluated, along withcorresponding efforts. Selecting and adapting components, for instance, may demand considerable effort. A completemodel for OO projects resource estimation that accommodates these concerns is our other goal.2.2 Strategy

The strategy toward the stated goals can be unfolded in several steps. Some of those steps can be “climbed” con-currently.

Steps for Goal 1:

i) MOOD metrics set proposal (first introduced in [Abreu94] );

ii) practical validation of the underlying rationale of the proposed set, by means of a comparative evaluation ofseveral “supposed-to-be-well-designed” OO systems (partly included in this paper);

iii) construction and public distribution of a tool for automatic collection of MOOD metrics from OO languagessource code (scheduled for this summer); support for industrial and academic wide experimentation5 andstatistical validation of results;

iv) theoretical validation of the MOOD metrics (using Measurement Theory);v) MOOD set refinement based on iii) and iv) results (MOOD V2 proposal);vi) embedding of MOOD V2 metrics set on a OO CASE tool;vii) assessment of correlation between MOOD metrics and maintainability sub-characteristics.Steps for Goal 2:

i) proposal of a generic OO software system complexity metric (under way);

ii) theoretical validation of the complexity metric (using a set of desiderata defined in [Weyuker88]);iii) develop a model for the effect of reuse in productivity, validated with published reports data;iv) proposal of a resource estimation model6;

v) construction and public distribution of a tool to support the estimation process, based on the proposed approach;vi) public validation of the model with real-world OO projects;vii) model calibration and refinement. 4 - A private Portuguese non-profit R&D organization

5 - Potential MOODKIT users will be asked to disclose the metrics collected with it (anonymity of origin will be guaranteed on request), thus helpingto enlarge the data set and calibrating the heuristics. Suggestions for tool improvement will be welcome..6 - To be named MOORED (Model for Object Oriented Resource Estimation Determination)

3. THE MOOD METRICS SET3.1 Introduction

The MOOD (Metrics for Object Oriented Design) set includes the following metrics:•Method Hiding Factor (MHF)•Attribute Inheritance Factor (AIF)•Attribute Hiding Factor (AHF)•Polymorphism Factor (PF)•Method Inheritance Factor (MIF)•Coupling Factor (CF)

Each of those metrics refers to a basic structural mechanism of the object-oriented paradigm as encapsulation (MHFand AHF), inheritance (MIF and AIF), polymorphism (PF) and message-passing (CF) and are expressed as quotients.The numerator represents the actual use of one of those mechanisms for a given design. The denominator, acting as anormalizer, represents the hypothetical maximum achievable use for the same mechanism on the same design (i.e.considering the same number of classes and inheritance relations). As a consequence, these metrics:

1. are expressed as percentages, ranging from 0% (no use) to 100% (maximum use);

2. are dimensionless, which avoids the often misleading, subjective or \"artificial\" units that pervaded the metricsliterature with its often “esoteric” flavor.Being formally defined, the MOOD metrics avoid subjectivity of measurement and thus allow replicability. In otherwords, different people at different times or places can yield the same values when measuring the same systems.

These metrics are also expected to be system size independent. A partial demonstration of this assertion is includedbelow. Size independence allows inter-project comparison, thus fostering cumulative knowledge.

The MOOD metrics definitions make no reference to specific language constructs. However, since each language hasits own constructs that allow for implementation of OO mechanisms in more or less detail, a binding for the casestudies language (C++) is included ahead7. Similar bindings will be proposed for other OO languages in the nearfuture. A validation experiment with Eiffel [Meyer92] is currently under way. This expected, but yet to be proved,language independence will broaden the applicability of this metric set by allowing comparison of heterogeneoussystem implementations.3.2 A supporting example

To help clarify the metrics determination process, the following C++ code, adapted from [Young92], will be used innext sections. It is a subset of a class taxonomy where Application and Clock inherit from UIComponent. This lastclass inherits from BasicComponent (the base class).class BasicComponent { protected:

char *_name;Widget _w;

public:

BasicComponent( const char *); //Constructor

public:

virtual ~BasicComponent(); // Destructorvirtual void manage();virtual void unmanage();

const Widget baseWidget() { return _w; }

};

class UIComponent : public BasicComponent { private:

static void widgetDestroyedCallback

( Widget, XtPointer, XtPointer );

protected:

UIComponent ( const char * ); //Constructor

void installDestroyHandler();

7 - There are still some pending issues related to how templates and exception handling (throw, try, catch, ...) should be considered.

virtual void widgetDestroyed();void setDefaultResources

( const Widget , const String *);

void getResources

( const XtResourceList, const int );virtual ~UIComponent();// Destructorvirtual void manage();

virtual const char *const className()

{ return \"UIComponent\"; }

};

class Application : public UIComponent {

// Allow main and MainWindow to access protectedmember functions

#if (XlibSpecificationRelease>=5)

friend void main ( int, char ** );

#else

friend void main ( unsigned int, char ** );

#endif

friend class MainWindow;

private:

void registerWindow ( MainWindow * );void unregisterWindow ( MainWindow * );

protected:

Display *_display;

XtAppContext _appContext;

// Functions to handle Xt interface#if (XlibSpecificationRelease>=5) virtual void initialize ( int *, char ** );#else

virtual void initialize(unsigned int *,char **);#endif

virtual void handleEvents();char

MainWindowint

public:

Application ( char * );// Constructorvirtual ~Application();// Destructorvoid manage();void unmanage();void iconify();

Display *display()

{ return _display; }

XtAppContext appContext()

{ return _appContext; }

const char *applicationClass()

{ return _applicationClass; }

3.3 Metrics definition and language bindings3.3.1 Method Hiding Factor:

*_applicationClass;**_windows;_numWindows;

};

virtual const char *const className()

{ return \"Application\"; }

class Clock : public UIComponent {

private:

int_delta;// The time between ticksXtIntervalId _id;// Xt Timeout identifier

virtual void timeout();// Called every deltavirtual void speedChanged ( int );

static void timeoutCallback

( XtPointer, XtIntervalId * );

static void speedChangedCallback

( Widget, XtPointer, XtPointer );

protected:

virtual void tick()= 0;

public:

Clock ( Widget, char *,

int// Minimum speedint );// Maximum speed

~Clock ();// Destructor

void stop();void pulse();void start();

// Stop the clock

// Make the clock tick once// Start or restart the clock

virtual const char *const className()

{ return ( \"Clock\" ); }

};

∑MHF=∑TCi=1TCi=1Mh(Ci)Md(Ci)where:

TC = total number of classes in the system under considerationMd(Ci)=Mv(Ci)+Mh(Ci)=methods defined in CiMv(Ci) = visible methods in class CiMh(Ci) = hidden methods in class Ci

MOOD/C++ bindings

• methods - constructors, destructors, function members (including virtual ones, also called deferred) and operatordefinitions

• visible methods - methods in public clause

• hidden methods - methods in private8 and protected9 clausesNotes:

• inherited methods are not considered here; 8 - These methods are only reachable within the scope of the class to which they belong (i.e. they are not even inherited by subclasses).9 - These methods are only reachable within the scope of the class to which they belong and their derived classes (descendants).

• function members with the same identifier (“function-name overloading”) but with different signatures (distinctformal parameter list) are considered as distinct methods.Examples:

• Mh(BasicComponent) = 1 { constructor }

• Mv(BasicComponent) = 4 { destructor,manage, unmanage, baseWidget }• Md(BasicComponent) = 1 + 4 = 5

• Mh(UIComponent) = 6 { widgetDestroyedCall-back, constructor, installDestroyHandler, widget-Destroyed, setDefaultResources, getResources }• Mv(UIComponent) = 3 { destructor, manage,className }

• Md(UIComponent) = 6 + 3 = 93.3.2 Attribute Hiding Factor :

• Mh(Application) = 5 { main, registerWindow,unregisterWindow, initialize, handleEvents }

• Mv(Application) = 9 { constructor, destructor,manage, unmanage, iconify, display, appContext,applicationClass, className }• Md(Application) = 5 + 9 = 14

• Mh(Clock) = 5 { timeout, speedChanged, time-outCallback, speedChangedCallback, tick }

• Mv(Clock) = 6 { constructor, destructor, stop,pulse, start, className }• Md(Clock) = 5 + 6 = 11

AHF=where:∑∑TCi=1TCi=1Ah(Ci)Ad(Ci)Ad(Ci)=Av(Ci)+Ah(Ci) = attributes defined in CiAv(Ci) = visible attributes in class CiAh(Ci) = hidden attributes in class Ci

MOOD/C++ bindings

• attributes - data members

• visible attributes - data members in public clause

• hidden attributes - data members in private10 and protected11 clausesExamples:

• Ah(BasicComponent) = 2 { _name, _w }• Av(BasicComponent) = 0

• Ad(BasicComponent) = 2 + 0 = 2• Ah(UIComponent) = 0• Av(UIComponent) = 0

• Ad(UIComponent) = 0 + 0 = 03.3.3 Method Inheritance Factor:

• Ah(Application) = 5 { _display, _appContext,_applicationClass, _windows, _numWindows }• Av(Application) = 0

• Ad(Application) = 5 + 0 = 5• Ah(Clock) = 2 { _delta, _id }• Av(Clock) = 0

• Ad(Clock) = 2 + 0 = 2

∑MIF=∑where:TCi=1TCi=1Mi(Ci)Ma(Ci)Ma(Ci)=Md(Ci)+Mi(Ci)= available methods in CiMd(Ci)=Mn(Ci)+Mo(Ci)=methods defined in CiMn(Ci) = new methods in Ci

Mo(Ci) = overriding methods in CiMi(Ci) = methods inherited in Ci

MOOD/C++ bindings

10 - These attributes are only reachable within the scope of the class to which they belong (i.e. they are not even inherited by subclasses).11 - These attributes are only reachable within the scope of the class to which they belong and their descendants.

• • • • •

methods defined - those declared within Ci

new methods - those declared within Ci that do not override inherited ones

overriding methods - those declared within Ci that override (redefine) inherited onesmethods inherited - those inherited (and not overridden) in Ci

available methods - those that can be invoked in association with Ci

• Mo(Application) = 3 { manage, unmanage,className }

• Mi(Application) = 9 { BasicComponent,~BasicComponent, baseWidget, UIComponent,installDestroyHandler, widgetDestroyed,setDefaultResources, getResources, ~UIComponent}

• Md(Application) = 11 + 3 = 14• Ma(Application) = 14 + 9 = 23

• Mn(Clock) = 10 { timeout, speedChanged, time-outCallback, speedChangedCallback, tick,constructor, destructor, stop, pulse, start }• Mo(Clock) = 1 { className }

• Mi(Clock) = 11 { BasicComponent,~BasicComponent, unmanage, baseWidget,UIComponent, installDestroyHandler, widget-Destroyed, setDefaultResources, getResources,~UIComponent, manage }• Md(Clock) = 10 + 1 = 11• Ma(Clock) = 11 + 11 = 22

Examples:

• Mn(BasicComponent) = 5 { constructor,destructor, manage, unmanage, baseWidget }• Mo(BasicComponent) = 0• Mi(BasicComponent) = 0

• Md(BasicComponent) = 5 + 0 = 5• Ma(BasicComponent) = 5 + 0 = 5

• Mn(UIComponent) = 8 { widgetDestroyedCall-back, constructor, installDestroyHandler, widget-Destroyed, setDefaultResources, getResources,destructor, className }

• Mo(UIComponent) = 1 { manage }

• Mi(UIComponent) = 4 { BasicComponent,~BasicComponent, unmanage, baseWidget }• Md(UIComponent) = 8 + 1 = 9• Ma(UIComponent) = 9 + 4 = 13

• Mn(Application) = 11 { main, registerWindow,unregisterWindow, initialize, handleEvents, cons-tructor, destructor, iconify, display, appContext, ap-plicationClass }3.3.4 Attribute Inheritance Factor :

AIF=where:∑∑TCi=1TCi=1Ai(Ci)Aa(Ci)Aa(Ci)=Ad(Ci)+Ai(Ci) = attributes available in CiAd(Ci)=An(Ci)+Ao(Ci)= attributes defined in CiAn(Ci)= new attributes in class Ci

Ao(Ci)= overriding attributes in class CiAi(Ci)= attributes inherited in class Ci

MOOD/C++ bindings

• attributes defined - those declared within Ci

• new attributes defined - those declared within Ci that do not override inherited ones• overriding attributes - those declared within Ci that override (redefine) inherited ones• attributes inherited - those inherited (and not overridden) in Ci

• available attributes - those that can be manipulated in association with CiExamples:

• An(BasicComponent) = 2 { _name, _w }• Ao(BasicComponent) = 0• Ai(BasicComponent) = 0

• Ad(BasicComponent) = 2 + 0 = 2• Aa(BasicComponent) = 2 + 0 = 2• An(UIComponent) = 0

• • • • •

Ao(UIComponent) = 0

Ai(UIComponent) = 2 { _name, _w }Ad(UIComponent) = 0 + 0 = 0Aa(UIComponent) = 0 + 2 = 2

An(Application) = 5 { _display, _appContext,

_applicationClass, _windows, _numWindows }• Ao(Application) = 0

• • • •

Ai(Application) = 2 { _name, _w }Ad(Application) = 5 + 0 = 5Aa(Application) = 5 + 2 = 7An(Clock) = 2 { _delta, _id }

• • • •

Ao(Clock) = 0

Ai(Clock) = 2 { _name, _w }Ad(Clock) = 2 + 0 = 2Aa(Clock) = 2 + 2 = 4

3.3.5 Polymorphism Factor:

PF=where:∑TCi=1∑TCi=1Mo(Ci)Mn(Ci)×DC(Ci)Mo(Ci) = overriding methods in class CiMn(Ci) = new methods in class Ci

DC(Ci) = number of descendants of class Ci(derived classes)

The numerator represents the actual number of possible different polymorphic situations. Indeed, a given messagesent to class Ci can be bound (statically or dynamically) to a named method implementation, which can have asmany shapes (“morphos” in ancient greek) as the number of times this same method is overridden (in Ci descen-dants).

The denominator represents the maximum number of possible distinct polymorphic situations for class Ci. This wouldbe the case where all new methods defined in Ci would be overridden in all its derived classes.

Note: this metric definition was somewhat modified, when compared with its initial proposal [Abreu94]; this was aresult of our better understanding, due to the experimental validation described in this paper.Examples:

• Mo(BasicComponent) = 0

• Mn(BasicComponent) = 5 { constructor,destructor, manage, unmanage, baseWidget }

• DC(BasicComponent) = 3 { UIComponent,Application, Clock }

• Mo(UIComponent) = 1 { manage }

• Mn(UIComponent) = 8 { widgetDestroyedCall-back, constructor, installDestroyHandler, widget-Destroyed, setDefaultResources, getResources,destructor, className }

• DC(UIComponent) = 2 { Application, Clock }3.3.6 Coupling Factor:

• Mo(Application) = 3 { manage, unmanage,className }

• Mn(Application) = 11 { main, registerWindow,unregisterWindow, initialize, handleEvents, cons-tructor, destructor, iconify, display, appContext,applicationClass }

• DC(Application) = 0

• Mo(Clock) = 1 { className }

• Mn(Clock) = 10 { timeout, speedChanged, time-outCallback, speedChangedCallback, tick,constructor, destructor, stop, pulse, start }• DC(Clock) = 0

COF=where:∑∑i=1TCTCj=1is_client(Ci,Cj)TCTC2−TC−2×∑i=1DC(Ci)TC2−TC = maximum number of couplings in a system with TC classes

TC

2×∑i=1DC(Ci) = maximum number of couplings due to inheritanceis_client(Cc,Cs)=

1iff

Cc⇒Cs∧Cc≠Cs∧¬(Cc→Cs)

0otherwise

The client-server relation (Cc⇒Cs) means that Cc(client class) contains at least one reference to a feature (methodor attribute) of class Cs(supplier class). Cc→Cs represents an inheritance relation (Cc inheriting from Cs).

Because we want to evaluate non-inheritance coupling, Cc and Cs should not be tied by any inheritance relationship(direct or indirect). The numerator then represents the actual number of couplings not imputable to inheritance. Thedenominator stands for the maximum possible number of non-inheritance couplings in a system with TC classes.Note: this metric definition was also refined, when compared with its initial proposal [Abreu94].

MOOD/C++ bindings

Client-server relations can have several shapes:

• regular message passing - a given class invokes another class interface function member;

• “forced” message passing - a given class invokes any type (visible or hidden) of function member from anotherclass by means of a friend clause;

• object allocation and deallocation - a class invokes another class constructor or destructor;

• reference (in the client class) to a server class as an attribute or within a method formal parameter list - thesereferences are due to semantic associations12 among classes;Example:

• UIComponent is client of class MainWindow4. A FIELD TRIAL4.1 The Tool

MOODKIT, a simplified tool for metrics extraction from source code, was developed and tested with the sampledescribed in the next section. Version 1.1 supports the collection on C++ code of all MOOD metrics mentioned inthis paper. It was built using ANSI C and scripts with standard UNIX commands (awk, grep, find, etc).

It is worth mentioning that the first attempt to collect the MOOD metrics (on the MFC library, described ahead) wasdone manually. It took an effort of about two man.weeks (two persons during a full week) and it became clear that thecollection process is really a repetitive, tedious, boring, time-consuming and expensive task for humans! By usingMOODKIT, the effort to do the same job was cut to a half man.day, something around 5% of the manual collectioneffort.

4.2 The Sample

The population to be modeled is not the whole set of software systems built using the OO paradigm. Instead, we wanta representation of a population of “well designed” ones. We believe that the chosen set meets this goal, eitherbecause their elements have been in the public domain or commercial use for some time (with a great number ofusers trying them and suggesting upgrades) or because they were made by trustworthy teams with proven experiencefrom academia and/or industry.

The sample measured with MOODKIT V1.1 was a collection of class libraries written in the C++ programminglanguage. For each of those libraries a small synopsis is included below.

• Microsoft Foundation Classes (MFC) Origin: Microsoft Corporation

Brief description: application framework designed for use in developing applications for Microsoft Windows;works in concert with the entire Microsoft Visual C++ development system, including the Visual WorkBench,AppStudio, AppWizard and ClassWizard; includes general-purpose classes for time, date, data-structures and fileI/O, architectural classes to support commands, documents, views, printing and help, high level abstractionsincluding toolbars, status bars, edit views, form views, print previews, scrolling views and splitter windows andsupports standard OLE user-interfaces and shared DLLs.

• GNU glib++ (GNU)

Origin: Free Software Foundation / Cygnus Support

Brief description: part of GNU’s public domain programming environment; contains general purpose classes formanipulating strings, lists, files, etc.

• ET++ library (ET++)

Origin: UBILAB / Union des Banques Suisses (Switzerland) 12 - with a given arity (1:1, 1:n or n:m)

Brief description: homogeneous library integrating user interfacing building blocks, basic data structures andsupport for object input/output with high level application framework components; it uses the terminology ofSmalltalk-80 collection libraries.

• NewMat library (NMAT)

Origin: Robert B. Davies (robert.davies@vuw.ac.nz - Victoria University - New Zealand)

Brief description: package intended for scientists and engineers who need to manipulate a variety of types ofmatrices using standard matrix operations; emphasis is on the kind of operations needed in statistical calculationssuch as least squares, linear equation solving and eigenvalues.

• MotifApp library (MOTIF)

Origin: Douglas A. Young [Young92]

Brief description: public domain library providing a set of C++ classes on top of OSF/MOTIF for manipulation ofmenus, dialogs, windows and other widgets; it allows to use the OSF/MOTIF library in a OO style. Because itsmain creation purpose was academic (example included in a book), it is the smallest library and perhaps the leastused (and reengineered) in “real-life” of all included in the sample. It is likely to generate some outliers.For a better perspective over the sample, Table 1 includes some size metrics for each library. Rows refer to the totalnumber of declared classes, total number of declared methods, total number of declared attributes (data members) andthe total number of lines of code.

MFCGNUET++NMatMotifTOTAL1288426286355953080147848128481991041760815198012576194074515960550221279884163556ClassesMethodsAttrib.LOCTable 1 - Some indicators of sample size

4.3 The Results

Table 2 and figure 1 summarize the results obtained through application of MOODKIT on the above sample. Eventhough the size of the sample is not as large as we would have liked, we consider it sufficiently meaningful for thepurpose of the remaining study. The statistical analysis included in the next sections is expected to partiallycorroborate this assumption.

MFC24,6%68,4%83,2%59,6%9,0%2,7%GNU13,3%84,1%63,1%62,6%2,8%3,5%ET++9,6%69,4%83,9%51,8%7,7%4,5%NMatMotif11,1%39,2%76,8%100,0%73,1%,3%56,6%50,3%27,1%7,6%12,2%9,8%MHFAHFMIFAIFCOFPFTable 2 - Resulting metrics for the sample

100%90%80%70%60%50%40%30%20%10%0%MHFAHFMIFAIFCOFPFMFCGNUET++NMatMotifFigure 1 - MOOD metrics

5. DISCUSSION

5.1 Which shape for design heuristics?

The MOOD metric set enables expression of some recommendations for designers. This section explains the cor-responding rationale. We will pick an Electronic Engineering analogy for representing our design heuristics. Let it becalled “the filters metaphor”.

Theoretically, a high-pass filter is not expected to affect signal frequencies above a certain value (the cutoff fre-quency). Below that value, the filter acts as a hindrance for frequency. By analogy, a high-pass heuristic is the onethat suggests that there is a lower limit for a given metric. Going below that limit is a hindrance to resulting softwarequality. For those who do not like thresholds, we may say that the analogy is even more perfect, if we realize that“real” filters do not have them. Indeed their shape is not a step but a curve with a bigger slope at the cutoff zone.Resulting software quality characteristics are also expected to be strongly attenuated (or increased, depending on thedirection) as we approach the cutoff values. The reasoning for a band-pass heuristic is similar, except that we havetwo cutoff zones (a lower and an higher one).

AHF and MHF are a measure of the use of the information hiding concept that is supported by the encapsulationmechanism. Information hiding allows, among other things, to: (i) cope with complexity by looking at complexcomponents such as “black boxes”, (ii) reduce \"side-effects\" provoked by implementation refinement, (iii) support atop-down approach, (iv) test and integrate systems incrementally.

For attributes (AHF) we want this mechanism to be used as much as possible. Ideally13 all attributes would be hidden,thus being only accessed by the corresponding class methods. Very low values for AHF should trigger the designers’attention. The corresponding design heuristic shape is that of a high-pass filter.

The number of visible methods is a measure of the class functionality. Increasing the overall functionality will thenreduce MHF. However, to implement this functionality we must adopt a top-down approach, where the abstractinterface (visible methods) should only be the tip of the iceberg. In other words, the implementation of the classinterface should be a stepwise decomposition process, where more and more details are added. This decompositionwill use hidden methods, thus getting the above mentioned information-hiding benefits and favoring a MHF increase.This apparent contradiction is reconciled if we consider MHF to have values within an interval. A very low MHFwould then indicate an insufficiently abstracted implementation. Conversely, a high MHF would indicate very littlefunctionality. The design heuristic shape for MHF is thus the same as for a band-pass filter.

MIF and AIF are measures of inheritance, a mechanism for expressing similarity among classes that allows the por-trayal of generalization and specialization relations and a simplification of the definition of inheriting classes, bymeans of reuse. At first sight we might be tempted to think that inheritance should be used extensively. However, thecomposition of several inheritance relations builds a directed acyclic graph (inheritance hierarchy tree), whose depthand width make understandability and testability quickly fade away. A band-pass filter shape seems appropriate forthe corresponding heuristics.

13 - Such is the case in the Motif library.

The COF metric is a measure of coupling between classes. Coupling can be due to message-passing among classinstances (dynamic coupling) or to semantic association links (static coupling). It has been noted [Meyer88] that it isdesirable that classes communicate with as few others as possible and even then, that they exchange as littleinformation as possible. Coupling relations increase complexity, reduce encapsulation and potential reuse, and limitunderstandability and maintainability. Thus, it seems that we should avoid it as much as possible. Very high values ofCOF should be avoided by designers. However, for a given application, classes must cooperate somehow to deliversome kind of functionality. Therefore, COF is expected to be lower bounded. Accordingly, the design heuristic shapewill be the one of a band-pass filter.

Resulting polymorphism potential is measured through the PF metric. Polymorphism arises from inheritance and itsuse has pros and cons. Allowing binding (usually at run time) of a common message call to one of several classes (inthe same hierarchy) is supposed to reduce complexity and to allow refinement of the class hierarchy without side-effects. On the other hand, if we need to debug such a hierarchy, by tracing the control flow, this same polymorphismwill make the job harder14. We can then state that polymorphism ought to be bounded within a certain range.Naturally, a band-pass filter is the corresponding shape for the respective design heuristic.

As a conclusion we may say that the design heuristics can exhibit two shapes: high-pass (HP) and band-pass (BP),depending on the metric considered, as shown in the next table.

Minimum10,4%70,2%66,2%52,4%3,9%3,5%MeanMaximumShape19,6%28,7%BP79,7%HP73,5%80,8%BP56,2%60,0%BP10,8%17,7%BP6,5%9,6%BPMHFAHFMIFAIFCOFPFTable 3 - 90% Confidence interval for the sample

5.2 Confidence intervals

The confidence interval is a range on either side of the mean. If the sample has a considerable size (which is not yetthe case), we can say with a particular level of confidence (90% in this case) that all the population metric values willlie in the specified intervals. Being more specific, if we assume that:a) the population has a normal distribution

b) the sample is a good representative of the population

then the probability that further randomly sampled metrics lie inside the corresponding intervals is 90%. Theaccuracy of the intervals (i.e. their range reduction) is proportional to the square root of the sample size. We can thenexpect to disclose more accurate ranges and/or a bigger confidence (e.g. 95%) depending on our sample growth.The first hypothesis (normal distribution) is a usual starting point for statistical analysis. Again, with a bigger sample,we may find that another type of distribution is a better representative. The second hypothesis (sample re-presentativeness) has already been agreed upon in section 4.

Table 3 represents the 90% confidence interval for the sample mean of each MOOD metric. Taking into account theconsiderations made regarding the heuristics shape made in the previous section, we can take as initial thresholds fortriggering the designer attention, the values in the shaded zones. For instance, if the Coupling Factor exceeds 17,7%the designer could be warned somehow (supposing that he is using a design tool with embedded metrics capture). Hewould then realize that his design lies outside the “normal” boundaries of good practice and that the consequencesmay be the ones already referred to. Besides this outlier identification, the MOOD metrics can also help choosebetween alternative design implementations by helping to rank them.5.3 Size independence

In this section we will analyze the hypothesis formulated about the size independence of each MOOD metric. For thatpurpose we correlated each of the sizes included in Table 1, with the corresponding values of each metric (in Table2). The resulting correlations follow: 14 - This is particularly true if we compare this situation with the procedural counterpart, where for a similar functionality we usually have a series ofdecision statements for triggering the required operation.

MHFAHFMIFAIFCOFPFClassesMethodsAttrib.-0,59-0,48-0,37-0,74-0,79-0,740,790,840,88-0,19-0,05-0,21-0,14-0,28-0,25-0,47-0,66-0,58LOC-0,19-0,810,0,14-0,21-0,70Table 4 - Correlation of MOOD with some size metrics

Examining Table 4 we can find that all metrics except AHF and MIF (shaded zone) are fairly size independent asthey show low correlations15 with all size metrics. These anomalies might indicate one of two possibilities:a) AHF and MIF are ill-defined as to what constitutes the desired size-independence;b) the sample is somehow biased due to its small size;

We believe that hypothesis b) is more likely to be true, mainly because AHF and MIF have similar definitions toMHF and AIF, respectively, which show no significant correlation with any measure of size. Therefore, AHF andMIF size-dependence can not be conclusive until a bigger sample is available and analyzed. As our sample grows, wewill wait until the correlations stabilize and then infer a more definite conclusion.5.4 Statistical independence

Each MOOD metric should quantify a distinct feature of an OO system. To achieve this goal they need to be inde-pendent from each other. Besides, we think it is possible to interpret MOOD metrics as probabilities16 in the sensethat they quantify the presence or absence of a certain feature. This kind of interpretation allows the application ofstatistical theory to the MOOD metrics. If their statistical independent is proven, they can be combined (e.g.multiplied) so that the result can still be interpreted as a probability. With these reasons in mind, we evaluated thecorrelation among the sample value series for the defined metrics. Table 4 summarizes the results achieved.

MHFAHFMIF

MHF0,68-0,34AHF-0,87MIFAIFCOFPF

AIFCOFPF-0,38-0,280,17-0,31-0,200,46-0,130,15-0,35

-0,09-0,460,75Table 5 - Correlation among the MOOD metrics

Considering Table 5 we can conclude that most metric pairs (except the two shaded) exhibit a low correlation value.The COF-PF correlation is due to the COF value for the NewMat library (27,1%) which is clearly an outlier. If itsvalue was, for instance, equal to the other 4 libraries COF average (6,8%), then the COF-PF correlation would be0,11!

Such an easy explanation for the AHF-MIF correlation was not found. We believe it is a coincidental situation, againstemming from a small sample size17.

15 - Considering a 70% threshold, which seems appropriate.

16 - A similar approach was used in the ESPRIT REBOOT project [Stalhane92].

17 - Note that these metrics (AHF and MIF) are the same that have shown anomalies on the previous section.

6. FUTURE WORK

6.1 MOODKIT evolution and CASE tools

A beta-test version (V1.2) of the MOODKIT tool, that will support the collection, storage and analysis of the MOODmetrics on C++ source code, is under construction and will soon be disclosed for public domain.

A completely reengineered version 2 is being designed. Its core (metrics definition dictionary, metrics storage, hu-man-machine interface) will be based on a language independent central repository with storage, retrieval andgraphical capabilities. It will use specific “stubs”, based on language parsers, for metrics capture from distinct OOlanguage source code. An Eiffel stub is being built, and a Smalltalk one is also planned for the coming year. MOODbindings for those languages will also be published in following papers.

There is an increasing interest from OO CASE tool makers in design metrics. Output from the ROSE tool (whichsupports the Booch method), for instance, is being used at Rational [Fay94] to derive object-oriented metrics.Following this trend we are currently working with OBLOG Software (a private Portuguese R&D company), to ex-tend their OBLOG CASE tool (supporting the OBLOG - OBject LOGic method [Sernadas91]) with design metricsand heuristics based on MOOD.6.2 Experimental validation

Using MOODKIT V1.0, we started an extensive evaluation of available systems (C++ class libraries) and tried toderive and refine some design heuristics based on the results (metrics) obtained. The public availability of a nextversion of this toolkit, will allow people either from industry or academia to replicate the experiment herein describedwith other OO systems. The increase of the sample size will allow us to divide it in percentiles, discard outliers andachieve better confidence intervals. The refined heuristics will again be published.

As already mentioned, we also expect to prove that the MOOD metrics are fairly implementation language inde-pendent. Size independence and statistical independence among metrics will also be further and further assessed bymeans of correlation analysis.6.3 Analysis metrics

Metrics should be collected and used to identify possible flaws as early as possible in the life-cycle, before too muchwork is spent based on them. It is a well known fact that the effort of correcting and recovering from those defectsincreases non-linearly with elapsed project progress since they were committed. Looking at the analysis instead ofdesign would then be a step forward towards cost-effectiveness. The object-oriented paradigm is supposed, at leasttheoretically, to allow a seamless analysis-design-coding transition. Many analysis and design methods have emerged[Champeaux92] in the past few years, with their own diagrammatic representations of differently named abstractionsrepresenting not-so-different basic concepts. This plethora gave birth to tools, such as ParadigmPlus or Object-Maker18, supporting multiple analysis and design methods. These tools map the information extracted from the dis-tinct diagrams used by those different methods into a common repository, thus allowing diagrammatic conversions.Therefore, despite the apparent diversity of OO analysis models, we think it is possible to define a common set ofmetrics for analysis, a “natural” evolution of the MOOD set.7. FINAL OVERVIEW

The adoption of the Object-Oriented paradigm is expected to help produce better and cheaper software. The mainstructural mechanisms of this paradigm, namely, inheritance, encapsulation, information hiding or polymorphism, arethe keys to foster reuse and achieve easier maintainability. However, the use of language constructs that support thosemechanisms can be more or less intensive, depending mostly on the designer ability. We can then expect ratherdifferent quality products to emerge, as well as different productivity gains. Advances in quality and productivityneed to be correlated with the use of those constructs. We then need to evaluate this use quantitatively to guide OOdesign. A validation experiment of a metric set named MOOD, suited for this purpose was presented in this paper.This set allows comparison of different systems or different implementations of the same system. Some design heu-ristics based on a filter metaphor were introduced. Their automatic attainment was achieved and discussed on asample of C++ libraries. It is hoped that those heuristics will be of some help to novice designers.

Object-orientation is not “the” silver bullet [Brooks86], but it seems to be the best bullet available today to face thepervasive software crisis. Keeping on the evolution track means we must be able to quantify our improvements.Metrics will help us to achieve this goal. 18 - [Darscht94] reports the intention to build an OO metrics collection prototype integrated with this tool.

REFERENCES

Abreu, F. Brito and Carapuça, Rogério, \"Candidate Metrics for Object-Oriented Software within aTaxonomy Framework\October 1993; selected for reprint in the Journal of Systems and Software, Vol. 23 (1), pp. 87-96,July 1994.

[Abreu94]Abreu, F. Brito and Carapuça R., \"Object-Oriented Software Engineering: Measuring and Control-ling the Development Process\Proceedings of the 4th International Conference on Software

Quality, ASQC, McLean, VA, USA, October 1994.

[Brooks86]Brooks, Frederick P. Jr., \"Essence and Accidents of Software Engineering\Proceedings of In-formation Processing 86, H.-J. Kugler (ed.), Elsevier Science Publishers B. V. (North Holland), IFIP

86, also published in IEEE Computer, April 1987.

[Campanai94]Campanai M. and Nesi P., \"Supporting O-O Design with Metrics\Proceedings of TOOLS

Europe'94, France, 1994.

[Cant94]S.N. Cant, B. Henderson-Sellers, and D.R. Jeffery, \"Application of cognitive complexity metrics to

object-oriented programs\

[Champeaux92]Champeaux, Dennis De and Faure, Penelope, \"A Comparative Study of Object-Oriented Analysis

Methods\Journal of Object-Oriented Programming, vol. 4, n. 10, pp. 21-33, March / April 1992.

[Chidamber94]Chidamber S. and Kemerer C., \"A metrics suite for object oriented design\IEEE Transactions on

Software Engineering, vol. 20, n. 6, pp. 476-493, June 1994.

[Darscht94]Darscht, Pablo, “Assessing Objects Along the Development Process” (submission 2), Workshop on

Pragmatic and Theoretical Directions in Object-Oriented Software Metrics, OOPSLA’94, Portland,USA, October 1994.

[Dumke95]Dumke, Reiner R., \"A Measurement Framework for Object-Oriented Software Development\

submitted to Annals of Software Engineering,Vol.1, 1995

[Fay94]Fay, Bill and Hamilton, Jim and Ohnjec, Viktor, “Position/Experience Report” (submission 3),

Workshop on Pragmatic and Theoretical Directions in Object-Oriented Software Metrics,OOPSLA’94, Portland, USA, October 1994.

[Hopkins94]Hopkins, Trevor P., \"Complexity metrics for quality assessment of object-oriented design\

SQM´94, Edinburgh, July 1994, proceedings published as Software Quality Management II, vol. 2:Building Quality into Software, pp. 467-481, Computational Mechanics Press, 1994.

[ISO9126]ISO/IEC 9126, Information Technology - Software Product Evaluation - Quality Characteristics and

Guidelines for their use, 1991.

[Lewis91]Lewis, John A. and Henry, Sallie M. and Kafura, Dennis G. : \"An Empirical Study of the Object-Oriented Paradigm and Software Reuse\" Proc. of ACM SIGPLAN Annual Conference on Object-Ori-ented Programming Systems, Languages, and Applications (OOPSLA'91), pp. 184-196, 1991.[Melo95]Melo Walcélio L., Briand Lionel and Basili Victor R., “Measuring the Impact of Reuse on Quality

and Productivity in Object-Oriented Systems”, Technical Report CS-TR-3395, University ofMaryland, Dep. of Computer Science, January 1995.

[Meyer88]Meyer B., Object-oriented Software Construction, Prentice-Hall International, 1988.[Meyer92]Meyer B., Eiffel: The Language, Prentice Hall International, 1992.[Stalhane92]Stalhane T. and Coscolluela A., \"Final Report on Metrics\Deliverable D1.4.B1, ESPRIT Project

5327 (REBOOT), February 1992.

[Sellers95]Henderson-Sellers, B., \"Identifying internal and external characteristics of classes likely to be useful

as structural complexity metrics”, Proceedings of 1994 Intern. Conf. on Object Oriented In-formation Systems OOIS'94, London, December 1994, Springer-Verlag, pp.227-230, London, 1995.

[Sernadas91]Sernadas, C. and Fiadeiro, J., “Towards Object-Oriented Conceptual Modelling”, Data and

Knowledge Engineering, vol.6, n.6, pp.479-508, 1991.

[Weyuker88]Weyuker E., \"Evaluating Software Complexity Metrics\IEEE TSE, vol.14, n.9, pp. 1357-1365,

September 1988.

[Young92] Young, D.A., Object-Oriented Program-ming with C++ and OSF/MOTIF, Prentice-Hall, 1992[Abreu93]