Completeness and Consistency of the System Requirement Speciﬁcation

—Although the System Requirement Speciﬁcation, as a ﬁrst formal and detailed document, is the base for the software project in classic software methodologies, there is a noticeable problem of assuring the completeness of this document. The lack of its completeness causes uncertainty of the project foundations. This was one of motivations for agile methodologies – if the SRS cannot be easily validated, if it can change in late project phases, then get rid of the SRS. Replace formal requirements with user stories . However user stories are also requirements - mostly functional requirements. As agile methodologies focus on functional requirements, it is easy to forget quality requirements. In this paper we show the impact of quality requirements analysis on functional requirements exploration. Although in our experiment we noticed considerable large functional requirements increment, we went further and examined the impact of SRS consistency on its completeness. The research has shown that the increment of the revealed requirements count may be almost three times greater, compared to the standard requirement speciﬁcation method.


I. INTRODUCTION
Starting the software project, the customers may be uncertain of their expectations, or may simply be unable to imagine the whole complexity of the software system.The developer's task at this very early stage of the project is to reveal as much of the requirements as is possible.In other words -to assure the completeness of the System Requirement Specification.However, there is a noticeable problem with the completeness of the SRS; with its measurement and event with its definition.The formal definition for the requirements completeness says that the SRS is complete when the "responses of the software to all realizable classes of input data in all realizable classes of situations is included" [1].This definition is not very usable, as we do not know the number of "all realizable classed of input data" and "all realizable classes of situations".Other definitions [2] are also unusable.This leads us to the problem: "how can we be sure that the requirement specification is complete without knowing what the complete requirement specification is?" [3].As it is an impossible situation, we can not determine the absolute completeness; we may only define some metrics of a relative completeness (as the relative increment of the elicited requirements) [4].
The whole system quality depends on the completeness of the requirement specification.We may find a huge set of quality metrics in [5].As we take a close look at these metrics, we may see that many of them are evaluated relatively to the number of functions described in the requirement specification.As we do not know the completeness of the requirement specification, we cannot be sure about the reliability of these metrics and the total quality of the system.
The uncertainty of the SRS completeness causes the risk of the requirements change during the development process [6].The risk is extremely high in the classic "waterfall" software development model and it is one of the reasons why the iterative and incremental models became much more popular [7].Often, an "agile" development process is taken [8], when the requirements are revealed in the user acceptance tests of the working (but incomplete) system.Although this model goes well in the normal situations, there may be some rare and critical conditions, which are hard to reveal based solely on the customer's claims.The author's research has shown, that the problem of the SRS completeness uncertainty may be resolved indirectly -with the measurement the SRS consistency.This measurement (in a graph theory sense) is easy, and has the large impact on the number of the revealed requirements.Using this method, the author has achieved the substantial increment of the requirements (almost three times) compared to the standard method of the requirement specification.
The term "consistency" has somehow fuzzy meanings in software engineering: we may understand it as a lack of contradiction between two, or more, requirements [9] [10]; and also as traceability, which means the ability to trace the requirements to the software solutions [11] [12].In this paper, we understand consistency as a degree of coupling between the requirements.To measure the degree of the internal coupling of the System Requirements Specification, it must be treated not only as a text document, but also in some quasi-graph form; where the vertices represent the requirements, and the edges represent the references between the requirements.The references between the requirements should be created when one requirement impedes the other (e.g. the "data backup" requirement impedes the "data restore" requirement).These are called "trace" references.They should also be created when two requirements complement each other (e.g."logging in" complements "logging off").Other logical references between requirements should be added "manually" during the requirements specification.
There may be several types of requirements: functional requirements, data requirements, and quality requirements.Some of the quality requirements touch the reliability of the system.System reliability depends on the resolution if critical situations which impede the functional requirements.Detection of unresolved critical situations (i.e.not resolved with functional requirements) leads to the assumption that some functional requirements may be missing.However, this assumption must be confirmed (or denied) in the detailed analysis of the SRS.

II. MEASURING THE SRS COMPLETENESS AND CONSISTENCY
To evaluate the SRS quality we need a set of precise and objective metrics.Traditionally, when an SRS document is text written, we may count some specific weak phrases (as "adequate", "not limited to") [13].Having the SRS document stored in the quasi-graph form, we defined other metrics of the SRS completeness and consistency.

A. Metrics and Measures
We distinguished metrics and measures in quality measurement.We treat a metric as a quality factor to be measured, and a measure as a method of the measurement.We divided measures between direct or indirect measures.We counted direct measures directly in the document quasi-graph storage; and we calculated indirect measures using some formulas.Every direct measure results in some absolute number counted directly by simple graph analysis (e.g. the number of "trace" references).These we called objective measures, as they may be objectively evaluated.Objective measures, although easy to evaluate, are insufficient to express a complete document quality.Some human analysis is needed to reveal the missing requirements, or to find inconsistent ones.The values resulting from this analysis are called expert measures, as the document review must be done by and expert.The review of the SRS should contain a list of missing requirements and a list of quality remarks to already defined requirements.As expert measures are less reliable than objective ones, the usage of expert measures is minimized.Two expert measures are used in this paper: one for completeness and one for consistency.Indirect measure expression usually divides two direct measure results.The result value is scaled from zero to one.Zero means the worst result and One means the best (ideal) result.When the numerator measure gives the number of "negative" elements (e.g.missing elements), then the function for an indirect measure is defined by a formula: where the Floor function rounds the first argument towards zero (with the precision of two decimal digits); m and n are direct measures.Rounding towards zero is needed as the result value of 1.0 can only appear in the ideal situation (e.g.no missing elements).
If one metric is evaluated with several measures, some aggregate function is needed, e.g. a weighted mean function: ..m n are component measures, and w 1 , w 2 , ...w n are their weights respectively.Some other terms used in metrics and measures definitions (below) need explanation.These terms are: element, metaclass, relationship, reference and solution.An element means an item of the software project (e.g. the system requirement).Each element has its meta-class, which describes its possible and required properties and relationships (e.g.FunctionalRequirement is one of the meta-classes).Relationships are not only references, which are meaningful only in the development process, but also associations meaningful in the runtime.Solution means here an element, or a set of elements, defined with a "trace" reference as the elaboration of some element.

B. SRS Completeness Metrics and Measures
Proposed metrics of the SRS Completeness (CP) are: Formal Completeness (FCP), Semantic Completeness (SCP) and Reference Completeness (RCP) -see fig. 1. Formal Completeness (FCP) involves Template Completeness Factor (TCPF) and Definition Completeness Factor (DCPF).First factor (TCPF) tells us, how completely a template of the document is filled.We count the number of elements required by a document template (the number of required meta-classes), and search missing ones.Second factor (DCPF) represents the number of elements with incomplete definition (with one, or more, properties required by meta-classes missing).
Semantic Completeness (SCP) uses an expert measure -Missing Semantic Element Count (MSEC).The missing elements must be listed by an expert revising the document.To get the value scaled from Zero to One we must divide MSEC not only by the Total Semantic Element Count (TSEC), but by the sum of TSEC and MSEC.
Reference Completeness (RCP) depends on two measures.First; a solution completeness factor (SLCF), evaluates the "trace" references leading to the "solutions".Second; an internal reference factor (IRFF), evaluates all missing references, that are required by the elements' meta-classes.

C. Consistency Metrics and Measures
Consistency of the SRS (CS) depends on two metrics: Formal Coherence (FCH) and Semantic Consistency (SCS) -see fig. 2.
Formal Coherence (FCH) is measured using graph-theory.When the document (SRS) is represented in a quasi-graph form, nodes represent project elements (requirements) and edges represent relationships (references).Although the references are directed, we evaluate Weak Coherence Factor (WCHF), which is appropriate for undirected graphs (edge direction is examined while measuring correctness -beyond this paper).WCHF is based on Weak Coherence Count (WCHC), which means a count of subgraphs which are Semantic consistency (SCS) depends on Semantic Consistency Factor (SCF), which is measured by an expert.The expert must judge if the semantic elements (requirements) are consistent with other elements, and mark inconsistent ones.

III. EXPERIMENT
A set of metrics and measures for completeness and consistency (shown above) was implemented in the IQuest system, which evaluated not only completeness and consistency, but also correctness, understandability, modifiability and verifiability.However we focus in this paper on the completeness and consistency, as these metrics showed the most interesting coincidence.
The IQuest system allowed us to import a sample SRS document written in Microsoft Office document format as an object-based requirements model where each requirement became an object and these objects were linked with references entered by a requirement manager.We used this object model to evaluate the consistency of a sample requirement specification.The IQuest system evaluated objective quality measures automatically, however an expert had to entered few data to evaluate the expert measures.

A. Evaluating the Quality of a Sample SRS Document
The sample SRS document for a Web Supermarket was elaborated for the research.The specification was presented as the set of requirement specification tables (see fig. 3).Note that references used to evaluate the SRS consistency are marked with '#' Fig. 3. Example of the functional requirement Three incrementing versions were prepared and their quality was evaluated.The completeness was decided (for the research) to be the most important factor for requirements specification.The correctness and consistency were decided to be less important and the importance of other metrics was only supplementary.The weights of 40, 20, 20, 10, 5, and 5 were assigned to the quality factors.All the metrics were scaled from 1 (worst) to 6 (best).
The objective measures was not only ones which led to new requirements definition.In the second version also a business expert was asked to find out missing requirements.

B. First version -no quality requirements defined
First version of the SRS document was prepared very thoroughly -as much as 69 functional requirements were specified.Although the quality requirements were not specified, its quality was evaluated.No expert was asked for evaluation because the specification was not finished at that moment.The results are shown in the tab.I.
As no quality requirements were specified, the low value of completeness agreed with expectations.

C. The second version -quality requirements consideration
Based on the first SRS, the next version with 36 quality requirements was developed.The quality requirements were revealed according to the template presented by table II.Exceptional, critical and breakdown situations were deliberated for reliability.For each such situation, a functional requirement was specified to prevent, or fix, this situation.That resulted in 99 new functional requirements.The results of the quality evaluation for the second version are shown in table III.
Despite expectation, completeness increased very little (from 3.1 to 3.65).Two explanations were discovered by the detailed result analysis (see table IV).First, it is impossible to achieve high completeness without filling the template completely (FCP).Second, the Reference Completeness (RCP) grew, but not satisfying.Despite intensive work, about 30% of elements were left without solution.

D. Third version -consistency consideration
In the third version, the formally "unresolved" goals; advantages, needs, tasks and problems, were deliberated.The emphasis was laid on increasing consistency.Some of the analyzed elements had already solutions in the form of functional requirements, but 30 new requirements had to be specified.The results of quality evaluations are shown in table V.
Although the increment of the consistency was very small (from 5.85 to 5.9), the completeness grew from 3.65 to 5.2.The main reason was the growth of Reference Completeness (RCP) from 4.44 to 5.9.

E. Relationship between consistency and completeness
Fig. 4a presents consistency impact on completeness in the three versions of the specification.This impact is significant (not only) in a mathematical aspect.More important is the impact of consistency on the number of functional requirements.As you can see (rys.4b), this number grew about 3 times (from 69 to 198).It means that without quality evaluation and without consistency increment about 2/3 of functional requirements would be omitted and the requirements specification would be incomplete.

IV. CONCLUSIONS AND FURTHER RESEARCH
Consistency measurement of the Software Requirement Specification requires a quasi-graph representation of this document; with nodes representing requirements, and edges representing references between requirements.The document     template defines permitted and required elements that should appear in the document.A set of objective measures may be defined to automate evaluation of the quality of the document.These objective measures are supplemented with expert measures, which are evaluated in the document review process.
The vast impact of consistency on completeness can be noticed while evaluating quality of the SRS document.Consistency does not guarantee completeness, but it may help to reveal much more requirements than using traditional methods.
Basing on this preliminary study we prepared the next experiment.At the first part of the experiment we collected about 50 SRS documents from developers which used the proposed method.Next we gave the same project subjects for other developers using agile methods.We plan to compare the number of functionalities discovered with traditional and with agile methods.However the results will be available after several months.

Fig. 4 .
Fig. 4. Consistency impact to completeness (a) and to the number of functional requirements (b) in the three versions of the specification.Dotted line at (a) shows the expected impact.

TABLE I QUALITY
RESULTS OF THE FIRST VERSION OF THE SPECIFICATION

TABLE II THE
STRUCTURE OF QUALITY REQUIREMENTS PART OF SRS

TABLE III QUALITY
RESULTS OF THE SECOND VERSION OF SPECIFICATION

TABLE V QUALITY
RESULTS OF THE THIRD VERSION OF SPECIFICATION