1. Introduction
In 2014, the Korean government declared the “Strategies to realize a Software-Oriented Society,” stressing that software should be at the center of innovation, growth, and value creation [1]. In particular, the development of embedded software source technology was declared important in industries such as aviation, automobiles, and railways. The Ministry of Public Administration and Security has accelerated the implementation of the master plan for safety innovation established in 2015 and has been carrying out field-oriented tasks such as realizing life-friendly safety policies, establishing preventive and initial response plans, and expanding disaster safety infrastructure [2]. On the other hand, the central government, local governments, and public institutions are performing detailed tasks and checking them periodically with the master plan for safety innovation.
As software safety-related systems and embedded software grow in scale and complexity, assuring safety is becoming more and more of an important issue [3-5]. Problems arising from software are not mere defects or malfunctions of services and products, but are becoming critical issues that can lead to enormous economic losses to the country, organization, and society and even further, loss of lives [6-9]. Given the recent cases of sudden, unintended acceleration crashes of cars and plane crashes, it is essential to ensure the functional safety of software in a safety-essential system. In order to ensure software safety in these safety-critical systems, it is necessary to improve the capabilities of software engineers.
However, there are some problems. First, the majority of software engineers are not mindful or fully aware of software safety and in many cases, lack expertise in various software engineering safety technologies. This is because of the characteristics of software, especially its invisibility and complexity, which means it does not have a physical form and is difficult to verify the final result during the development process [10,11]. Second, there is a shortage of personnel trained in software quality. Since it is the result of intellectual labor of human resources, the quality of the result depends on the expertise of that human resource, and therefore a more systematic and professional approach is required. However, according to [12] the software industry's human resource supply and demand status, the number of domestic IT quality engineers is quite small (2.6%), and the training for incumbent employees related to software quality and testing is only 1.7% [13].
Therefore, it is necessary to systematically develop software safety education programs. This research suggests high priority elements for software safety education programs in the common competency area based on needs analysis, in order to enhance software engineers’ safety awareness and expertise.
2. Literature Review
2.1 Prior Research on Software Safety
The analysis targets are as follows. Chin et al. [14] developed a software safety-related education curriculum proposal by investigating the roles and skills necessary for developing safety essential software and investigating the expectations of the end-users. In addition, the authors [15] designed a software safety professional qualification system consistent with international safety standards by investigating the required software safety activities from various perspectives, including development planning, design measures, test strategies, product management strategies, etc., based on IEC 61508, ISO 26262, DO-178B/C and EN 50128. Telecommunications Technology Association developed Software Safety Inspection Guide to improve the reality that national institutions have difficulty in diagnosing software safety, with the aim of ensuring national safety and enhancing safety [16]. Universities such as Southern California and Johns Hopkins offer educational curriculums for each safety activity such as safety requirements, safety analysis, safety design, and safety testing [14].
As such, an educational curriculum for software safety, a knowledge system for qualification systems, and safety guidelines at industrial sites have been proposed in accordance with international requirements. However, in the case of results dealing with the system and scope of knowledge, no specific derivation process is presented, and there is a tendency to focus on opinions among experts. First of all, the software safety elements derived from the above research results were defined at a higher level.
Therefore, results should be derived through more objective procedures, and the approach for this process needs to be more systematic. In this research, we collected and reviewed software safety-related researches from various sources including research institutes, associations, and international or industry standards to list potential educational elements with high priority.
2.2 Educational Needs Analysis
When developing an education program, the process of design, development, implementation, evaluation, and improvement is performed based on an analysis of educational needs. The program development process represented by the instructional systems design (ISD) model also shows the importance of analysis. For example, which content is extracted from the analysis, the first step in the ADDIE (analysis, design, development, implementation, evaluation) model, determines the process of the training program and the success of the evaluation. According to Sork [17], educational needs come from the difference between the current and desirable levels of ability, including knowledge, skills, and attitudes that can change through education or learning opportunities.
The analysis of educational needs is important, and the needs can be identified through either quantitative analysis or qualitatively through literature and by methods such as surveys. A survey is the most commonly used method for identifying educational needs [18]. According to the research trends in needs analysis studies identified by Cho [19], studies on needs analysis show that there is not much effort being made in sorting priorities. In addition, a more systematic method is required because the required analysis is often a numerical comparison, such as a presentation of the average or proportion of items.
By reviewing the various methods of demand analysis that have been used as individual methods in various studies, Cho [20] suggested a packaged process of deriving education priorities using a t-test, the Borich needs assessment, and the Locus for Focus model. The process of applying the three different analysis methods can be described in five major steps.
Step 1: t-test to see the difference between “what is” and “what should be.”
Step 2: Borich needs assessment formula to set priority.
Step 3: Coordinate plane results presented through the Locus for Focus model.
Step 4: Identify the number of items in the HH segment (high importance/high discrepancy) of the Locus for Focus model and determine how many Borich needs are ranked with high priority.
Step 5: Borich needs determine the top and bottom groups through identification of items that are redundant between the top priority items in the formula and the HH segment items in the Locus for Focus models.
By using the three different methods simultaneously, it is possible to supplement the deficiencies of each method and provide comprehensive information for better decision making.
3. Research Method
3.1 Research Subject and Data Collection
An online survey was distributed to workers and students who either have work experience in the software or safety related industry or have participated in trainings or seminars held by the Telecommunications Technology Association in South Korea up to August 2020. The online survey was conducted using Google-Forms for 2 weeks from September 17–18, 2020, and emails to encourage cooperation were sent out twice during the survey period. Of the total 2,978 questionnaires distributed, 281 copies (9.4% response rate) were collected, and finally, 259 copies (8.7% effective response rate) were used for the analysis, excluding 22 copies that were answered unfaithfully. The demographic information of the research subjects is described under Table 1.
Demographic information of research subjects
3.2 Research Process and Tools
The purpose of this study was to suggest high priority elements for the common software safety education program. To this end, a needs analysis proposed by Cho et al. [21] was performed to derive candidate elements and grasp priority. First of all, the existing literature on software safety was reviewed, and the elements of a common education program in the software safety field were derived by applying nominal group technique (NGT). Based on the results from the literature review and NGT, an online survey was conducted to identify the gap between “what is” and “what should be” in order to derive the priorities for education needs. For the priority analysis, the t-test, Borich needs assessment, and Locus for Focus model were employed to identify the final priorities of the education elements. The process in this research is shown in Fig. 1.
3.3 Validity of Survey Components
In order to secure the validity of the survey elements of this research, the degree of agreement was confirmed as to whether 74 potential educational components derived through the NGT and literature are suitable as the contents of the common educational program in the field of software safety. A questionnaire was conducted on 14 researchers in the QA/QC, test, and certification field of the software safety-related job in South Korea. According to Lawshe [22], the minimum content validity ratio (CVR) value of the components for 14 participants is 0.51. In the calculated result, there was one component (coding standard, CERT/CWE) that did not meet this the criteria. Although one element was not recognized for its validity, it was included in the needs analysis survey in reflection of several expert opinions that it is necessary for the overall composition. The potential 74 elements to be used in further surveys are shown in Table 2.
Potential education components result
3.4 Data Analysis
Analysis of the data collected through the survey was conducted as follows. First, a paired t-test was conducted to see if there is any significant difference between “what is” and “what should be.” Next, after applying Borich needs formula and the Locus for Focus model, the prioritization process proposed by Cho [20] was applied, and as a result, the education components with the highest priority for common software education programs in the software safety field were derived. First, the formula of Borich needs assessment, which was used as a method to determine priority, is as follows.
where RCL is required competence level, PCL is present competence level, and N is total number of cases.
The Locus for Focus model used in this study is a method of allocating the elements into quadrants with the level of necessity as the horizontal axis and the degree of discrepancy between “what is” and “what should be” as the vertical axis, showing priority as a result. Among the four quadrants, the first quadrant has a high degree of necessity and high discrepancy, making it the highest priority area for educational components.
4. Results
4.1 Result of t-test, Borich Needs Assessment
As shown in Table 3, there was a statistically significant difference between “what is” and “what should be” for the common education program elements in software safety areas (p < 0.001). As a result of Borich needs value assessment, it was confirmed that the rest of the elements except for the “definition of SW(a1)” were the highest, in the order of a2, a6, a3, a4, a5 followed by the malfunction identification criteria (d5), ISO 13482 standard (b10), safety integrity level (SIL) rating determination (d8), and hazard identification (d6).
Next, the result of analyzing priority using The Locus for Focus model is shown in Fig. 2. The average level of “what should be” perceived by the survey respondents, who are potential participants of the education, was 3.90 and the level of discrepancy was 1.08. These two averages were divided into axes, and out of the four quadrants, 21 elements were included in the first quadrant with the highest priority.
Lastly, the components with the highest priority in Borich were compared with the 21 elements located in the first quadrant in the Locus for Focus model for redundancy, as shown in Table 4. A total of 16 elements were derived as top priority items, which are in detail, the concept of software and safety and functional safety (a2, 3, 4, 5, 6), establishment of a standard process (b3), ISO 26262 standard (b5), safety integrity concept (d3), malfunction identification (d5), hazard identification (d6), hazardous event definition (d7), architecture types and dependencies (f3), architecture design principles (f4), safety mechanism design techniques (f8), fault injection (h6), and functional safety plan establishment (i2).
Result of Locus for Focus model.
Result of need analysis, t-test, Borish needs assessment
Top priority education elements for software safety education program
5. Discussion and Conclusion
In this study, a needs analysis was conducted to grasp the priority of necessary educational elements for the development of a common education program in the field of software safety, and the discussion based on the research results is as follows.
First, as a result of this study, 16 top-priority educational elements for software safety education programs have been derived. If an educational program needs to be organized within limited time and resources, the derived educational elements need to be reflected first. However, this does not mean that the entire software safety education program should be organized only with the derived high-priority educational elements, and other various factors such as the purpose of the program or the main target industry should be considered.
Second, looking at the 16 derived educational elements, we found that the concepts and definitions of software safety and functional safety are important for common field education programs and thus have high priorities. This study was conducted to develop educational programs that reflect the common characteristics of various industrial sectors at the foundation level. With this consideration, we confirm that the overall concept and understanding need to be reflected in the program in an in-depth manner.
Third, this study is designed as a systematic process and is a practical case of looking at high priority educational elements in the field of software safety through quantitative and qualitative research results. This can serve as an important reference for developing education programs in different levels or industries, and it can be applied in various fields dealing with knowledge, including the design of qualification systems and the establishment of knowledge.
Lastly, this work shows the result of a needs analysis to present top priority training elements before implementation of educational programs. The practical process from designing, implementing, operating, and evaluating the development of educational programs has not been presented in this study, which remains to be the limitation of the study. The mid-course results or post-training evaluation should also be analyzed and reflected for further improvement of the program, and through this process, the enhanced software safety education programs will surely lead to higher educational effectiveness.