Samsung Electronics produces various kinds of electronic devices such as memory, digital TV, printers, mp3 players, notebooks, mobile phones, network devices, home appliances, etc. Just as software is growing in size and complexity, development efforts are increasing tremendously. Recognizing this trend and the related importance of software testing, Samsung is striving to improve test methods and frameworks.
Along with the attempt to improve software quality, Samsung has previously tried various kinds of test scripts. However, due to a high diversity of different test languages scattered across the organization, communication and cooperation between test system developers are becoming increasingly difficult.
Following the interest in harmonizing Samsung's test system, our test managers proposed a unified test solution being used throughout the entire organization.
After evaluating different test methods, we have found TTCN-3 suitable to meet this challenging requirement. Our project aimed at gaining solid knowledge and understanding of TTCN-3 technology and the variety in terms of its application. For a lack of experience with TTCN-3, Samsung decided to found a partnership with Testing Technologies. Three pilot test systems were set up:
- A web service based test system using message-based communication
- A DTV test system using procedure-based communication
- A network base station test system using message-based communication
Samsung sets a high value on standards-based testing. We needed an all-in-one tool for TTCN-3 editing, compiling and debugging. The Java codec and adapter had to be included in the tool, providing a Java editor, compiler and debugger. Samsung wanted to purchase a reliable and flexible platform with a complete, integrated test management system to execute written test cases. In order to start using the tool as soon as possible, it had to be comprehensible and user-friendly.
TTworkbench complies with all requirements Samsung demanded, containing a comprehensive test management with an easy-to-use graphical editor. The availability of basic codecs and adapters saved us a lot of time and effort. Since the Eclipse-based TTCN-3 IDE is easy to extend, it ensures fast access to multiple technologies, which was important to us. Above all, TTworkbench provides full test automation features saving us further time and costs. Along with a convincing tool, we relied on Testing Technologies' consultancy services to assist us during the initial phase of our project.
Our Systems under Test (SUTs) consist of various kinds of above mentioned electronic devices with several Implementations under Test (IUTs) according to the test purposes. In our pilot test systems, we focused on specific layers: For the initial one, a web service system, the IUT was an in-house remote management protocol. It was followed by setting up a DTV test system with the IUT being a user command control. Finally, within a network test system, the IUT is processed by calls between different base stations.
Prior to engaging consultancy services of Testing Technologies we were only able to understand a few samples of TTCN-3 test cases. We knew how to modify them but did not manage to create new ones for SUTs which use different protocols. Furthermore, we did not exactly understand the test system architecture, codec (TCI), adapter (TRI), etc. Thanks to Testing Technologies' highly competent consultants, we are now able to design test systems for various kinds of SUTs using different protocols for a huge variety of test purposes.
So far we have performed this pilot project and have acquired basic knowledge about TTCN-3 test systems. We are now going to make our product test systems work in order to achieve best practice in the network domain. In a following phase, we will transfer the technology to an even wider scale of product tests and domains.
Samsung was extremely satisfied with the fast reaction time to our queries and the high quality of work delivered by Testing Technologies' test experts. They quickly understood our questions, were able to perceive issues from a customer's point of view, thus providing good solutions for all given tasks during the consultancy phase.
|
|
"The challenging task of applying a unified test solution in our entire organization requires a flexible test technology and tool. For this, TTCN-3 and TTworkbench have proven to be perfectly suitable. Samsung feels confident to successfully harmonize its test system and to achieve a clear improvement of software quality. The excellent consultancy service of Testing Technologies has helped us to quickly understand TTCN-3 and TTworkbench. We are now able to manage complex test scenarios on our own." |
Samsung's Software Quality Assurance Group |
