In the previous issue, we introduced common software quality measurement models (McCall, Boehm, ISO 9126 models). These models can be used to scientifically evaluate software quality.
This issue mainly introduces 7 software quality evaluation indicators (coding standards, source lines of code, bug rate per thousand lines of code, cyclomatic complexity, code coverage, fan-in/fan-out counts, design and development constraints).
1 Coding Standards
Measurement Standard: MISRA C/MAAB
Function: Standardization of coding/modeling
Regarding coding standards, we have introduced MISRA C and MAAB in previous issues; detailed introductions can be found in related articles on our public account. You can also experience MISRA C and MAAB related content through MATLAB’s Polyspace and Simulink modules (refer to relevant examples in MATLAB’s help documentation).
Example of MISRA C usage in MATLAB
Example of MAAB usage in MATLAB
2 Source Lines of Code (SLOC)
Measurement Standard: Number of source lines of code
Function: Workload assessment, coding efficiency evaluation
Source lines of code (SLOC) is a method of measuring program size by counting the number of lines of source code. This method may be the simplest software measurement metric, primarily reflecting software scale, commonly used to predict the workload needed for program development, and also to evaluate coding efficiency.
2.1 Two Types of SLOC Measurement Methods
Physical Lines of Code (LOC, Physical SLOC) are the number of lines of code in the program source code (including comment lines).
Logical Lines of Code (LLOC, Logical SLOC) measure the number of executable “statements” (different programming languages have different definitions).
2.2 Issues with SLOC Usage
When using the SLOC method for evaluation, ambiguities may arise. For example, in C code (Example 1):
for (i = 0; i < 100; i++) printf("hello"); /*How many lines are there?*/
The above code has 1 physical line of code (only 1 line), 2 logical lines of code (the for statement and the printf statement), and 1 comment line (1 line of comments).
Due to different coding habits and standards among programmers, the above code can be rewritten (Example 2):
/* How many lines are there now?*/for (i = 0; i < 100; i++){printf("hello");}
Now there are 5 physical lines of code, 2 logical lines of code, and 1 comment line.
From the above two examples with the same effect, it can be seen that there is a significant difference in the code line parameters. If comparison is needed, logical lines of code can be chosen for comparison to obtain relatively accurate information.
2.3 Advantages and Disadvantages of SLOC Method
Advantages: Effectively estimate workload, easy to count and intuitive.
Disadvantages: Cannot effectively assess programmer efficiency, cannot evaluate program functionality, may increase redundant code, does not consider differences between programming languages.
Practice has shown that the relationship between SLOC and workload is highly correlated; simply put, the larger the SLOC value, the longer the program development time, making it very effective for workload estimation. Since lines of code are a physical entity, they can be counted and statistically analyzed by programs, making it easy to visualize and obtain an intuitive measurement.
However, the SLOC measurement method has some controversies, especially since it can sometimes be misused. The SLOC measurement method does not effectively measure programmer efficiency. First, merely using the results of the coding phase to measure project productivity is ineffective, as the coding phase usually only accounts for 30% to 35% of the total workload; second, the correlation of the SLOC measurement method with program functionality is poor, as an excellent programmer may write only a few lines of code but be more efficient in functionality than a programmer who writes more lines; moreover, particularly experienced programmers are often assigned the most difficult tasks, and thus may sometimes appear “less efficient” than other programmers on a task; additionally, pursuing higher SLOC can motivate programmers to write unnecessary verbose code, increasing program complexity and maintenance costs; furthermore, in today’s software landscape, software is often developed using more than one language. Different languages use different numbers of lines of code to achieve the same functionality, making effective measurement of programs using SLOC impossible.
3 Bug Rate per Thousand Lines of Code
Measurement Standard: number of bugs/(lines of code/1000), the smaller the value, the better the quality
Function: Bug statistics/tracking/fixing
To achieve better testing and higher maintainability, bug tracking is essential. Bugs within each code segment, module, or time period (day, week, month, etc.) can be easily counted using tools. This allows for early detection and timely fixing.
Information about bug rates in CMMI level is as follows:
*CMMI (Capability Maturity Model Integration For Software) is a software capability maturity assessment standard developed over four years by experts from around the world in software process improvement and software development management organized by the Software Engineering Institute (SEI) at Carnegie Mellon University, and is widely promoted and implemented globally, mainly used to guide improvements in software development processes and assess software development capabilities.
3.2 Advantages and Disadvantages of Bug Rate
Regarding the bug rate per thousand lines of code, from an assessment standard perspective, a smaller bug rate indicates better quality. However, based on this result, it may lead team members to engage in behaviors that are not beneficial for long-term and overall efficiency, such as: increasing the base number, adding meaningless code; splitting fixed-length loops into sequential methods; hardcoding configurable information into the code; and excessive copying and pasting of code.
Although the bug rate per thousand lines of code does not explicitly encourage increasing the number of lines of code, this calculation result is quite unfair to excellent employees. It implicitly promotes the idea of “try to increase the number of lines of code”.
4 Cyclomatic Complexity
Measurement Standard: Size of cyclomatic complexity
Function: Measure structural complexity
Cyclomatic complexity is a measure of code complexity. In the concept of software testing, cyclomatic complexity is used to measure the complexity of a module’s decision structure, expressed numerically as the number of linearly independent paths, that is, the minimum number of paths required to reasonably prevent errors. A high cyclomatic complexity indicates that the program code may be of low quality and difficult to test and maintain.
4.1 Principle (Number of Branches)
A cyclomatic complexity of 1 means that there is only one path in the code; for code with one branch, its cyclomatic complexity is 2. A simple calculation method: start from 1 and continue down through the program. Each time a key word or similar term is encountered (if, while, repeat, for, and, or), increment by 1. In case statements, increment by 1 for each case.
4.2 Calculation Method Based on Flowchart
Formula 1: V(G)=e-n+2p. Where e represents the number of edges in the flowchart, n represents the number of nodes in the flowchart, and p represents the number of connected components in the flowchart (the number of connected components is the largest set of connected nodes). Since flowcharts are connected, p is 1.
Formula 2: V(G)=number of regions=number of decision nodes+1. In fact, the calculation of cyclomatic complexity can be done more intuitively, as cyclomatic complexity reflects the number of “decision conditions”. Therefore, cyclomatic complexity is essentially equal to the number of decision nodes plus 1, which is the number of regions in the control flow graph.
Formula 3: V(G)=R. Where R represents the number of regions that the control flow graph divides the plane into.
When calculating cyclomatic complexity for a program’s flowchart, it is best to use Formula 1; for a module’s flowchart, counting the number of decision nodes (Formula 2) is simpler; for complex flowcharts, using Formula 3 to calculate the number of regions is easier.
5 Code Coverage
Measurement Standard: The greater the coverage, the better
Function: Measure testing effectiveness
Code coverage is a measure of the extent of coverage of the code.
There are many ways to measure code coverage; here we introduce the four most common.
5.1 Statement Coverage
Also known as line coverage, segment coverage, basic block coverage, this is the most common and widely used coverage method, which measures whether each executable statement in the tested code has been executed.
Statement coverage is often criticized as the “weakest coverage” because it only covers executable statements in the code without considering various combinations of branches, etc. Testing effectiveness is not obvious, and it is difficult to discover more problems in the code.
int Example1(int a, int b){ return a / b;}
The test result shows that the code coverage reached 100%, but the simplest bug was not discovered: when b = 0, a division by zero exception occurs.
5.2 Branch Coverage
Also known as decision coverage, all-edges coverage, basic path coverage, decision-decision-path coverage.
Branch coverage measures whether each branch of every decision in the program has been tested. For example, the following code:
int Example2(int a, int b){ int Result = 0; if (a < 10 || b > 10) // Decision { return Result += 1; // Branch one } else { return Result -= 1; // Branch two } return Result;}
When designing branch coverage test cases, only consider the two outcomes of true and false for the decision, thus designing the following test cases can achieve 100% decision coverage:
TestCaes1: a = 5, b ＝ 5 // Decision result is true, Result = 1, covers branch oneTestCaes2: a = 15, b = 5 // Decision result is false, Result = -1, covers branch two
It measures whether the result of each sub-expression in the decision has been tested for both true and false. Using Example 2 code as an example:
int Example2(int a, int b){ int Result = 0; if (a < 10 || b > 10) // Decision { return Result += 1; // Branch one } else { return Result -= 1; // Branch two } return Result;}
Test cases designed based on condition coverage need to cover all possible outcomes of the condition expression; if there are n conditions, the number of test cases required is 2 raised to the power of n. In the example code, there are 2 conditions (a<10 and b>10), so 4 test cases are needed, specifically:
TestCaes1: a = 5, b ＝ 15 // a: true, b: true, decision is true, Result = 1TestCaes2: a = 5, b ＝ 5 // a: true, b: false, decision is true, Result = 1TestCaes3: a = 15, b = 15 // a: false, b: true, decision is true, Result = 1TestCaes4: a = 15, b = 5 // a: false, b: false, decision is false, Result = -1
In code with a large number of conditions, the number of test cases for condition coverage can be staggering; if there are 10 conditions, 2 raised to the power of 10, or 1024 test cases are needed to achieve 100% coverage.
5.4 Modified Condition/Decision Coverage (MC/DC)
Modified Condition/Decision Coverage (MC/DC) requires that each condition must independently affect the decision outcome. It is an optimization of condition coverage that can reduce the number of required test cases.
① Each input-output in a program must appear at least once;
② Each condition in the program must produce all possible output results at least once;
③ Each condition in every decision must independently affect the outcome of that decision, meaning that the value of that condition can be changed to alter the decision result while keeping other conditions unchanged.
The concept of MC/DC can be a bit convoluted, so let’s clarify it with specific code, still using Example 2 code:
int Example2(int a, int b){ int Result = 0; if (a < 10 || b > 10) // Decision { return Result += 1; // Branch one } else { return Result -= 1; // Branch two } return Result;}
In 5.3, the test cases that achieve 100% condition coverage are:
TestCaes1: a = 5, b ＝ 15 // a: true, b: true, decision is true, Result = 1TestCaes2: a = 5, b ＝ 5 // a: true, b: false, decision is true, Result = 1TestCaes3: a = 15, b = 15 // a: false, b: true, decision is true, Result = 1TestCaes4: a = 15, b = 5 // a: false, b: false, decision is false, Result = -1
The results of the condition coverage expressions are:
TestCaes1: (True || True) True
TestCaes2: (True || False) True
TestCaes3: (False || True) True
TestCaes4: (False || False) False
Based on the condition coverage test cases, the test cases designed according to MC/DC are as follows:
TestCaes2: a = 5, b ＝ 5 // a: true, b: false, decision is true, Result = 1TestCaes3: a = 15, b = 15 // a: false, b: true, decision is true, Result = 1TestCaes4: a = 15, b = 5 // a: false, b: false, decision is false, Result = -1
The number of test cases designed for MC/DC coverage is 3, covering the conditions:
TestCaes2: (True || False) True
TestCaes3: (False || True) True
TestCaes4: (False || False) False
The designed MC/DC test cases include program output results of 1 and -1, meeting requirements ① and ②. Comparing TestCase2 and TestCase4, b remains unchanged, and changing a results in opposite outcomes; comparing TestCase3 and TestCase4, a remains unchanged, and changing b results in opposite outcomes, meeting requirement ③. Therefore, MC/DC coverage reaches 100%.
For condition coverage in TestCase 1 and TestCase2, when the expression a < 10 is true, the decision result of if( ) will not be altered by the result of the second expression, it will still be true. Therefore, either condition coverage’s TestCase 1 or TestCase2 can be ignored, achieving a reduction in test cases.
Because MC/DC includes branch coverage in logical judgments and reduces the number of test cases based on condition coverage, it is recommended for practical use.
6 Fan-In/Fan-Out Counts of Functions/Modules
Measurement Standard: High fan-in, reasonable fan-out (3~4, 7≤)
Function: Measure complexity/reusability of functions/modules
The number of other modules called by a module is called the fan-out of that module. The larger the fan-out, the more issues need to be considered when designing that module, thus increasing complexity.
The number of other modules that call a module is called the fan-in of that module. A larger fan-in generally does not affect the complexity of the problem, and a larger fan-in indicates better reusability of that module.
To control the complexity of modules, the fan-out of a module should not be too large, generally not exceeding 7.If a module’s fan-out is found to be large (as shown in Figure 1), it may be necessary to consider re-decomposing (for example, changing to the scheme in Figure 2).
7 Design/Development Constraints
There are many design constraints and principles in software development, including:
-
Length of classes/methods
-
Number of methods/properties in a class
-
Number of parameters in methods/constructors
-
Use of magic numbers and strings in code files (magic numbers refer to specific values written directly in the code, making it difficult for others to understand the meaning of the numbers)
-
Proportion of comment lines, etc.
The maintainability and readability of code are very important. Development teams can choose one or all of these principles and follow them through some automation tools, which will greatly improve the quality of software products.
This concludes this issue’s content. We welcome everyone to leave comments and discuss in the comment area~
