{"id":12134,"date":"2022-04-20T09:52:22","date_gmt":"2022-04-20T06:52:22","guid":{"rendered":"http:\/\/journals.khnu.km.ua\/vestnik\/?p=12134"},"modified":"2022-04-20T13:51:28","modified_gmt":"2022-04-20T10:51:28","slug":"metamorphic-testing-as-a-service-a-new-design-pattern-of-cloud-serverless-systems-for-metamorphic-testing","status":"publish","type":"post","link":"https:\/\/journals.khnu.km.ua\/vestnik\/?p=12134","title":{"rendered":"Metamorphic testing-as-a-service: a new design pattern of cloud serverless systems for metamorphic testing"},"content":{"rendered":"

<\/p>\n

METAMORPHIC TESTING-AS-A-SERVICE: A NEW DESIGN PATTERN OF CLOUD SERVERLESS SYSTEMS FOR METAMORPHIC TESTING<\/p>\n

METAMORPHIC TESTING-AS-A-SERVICE: \u041d\u041e\u0412\u0418\u0419 \u0428\u0410\u0411\u041b\u041e\u041d \u041f\u0420\u041e\u0404\u041a\u0422\u0423\u0412\u0410\u041d\u041d\u042f \u0425\u041c\u0410\u0420\u041d\u0418\u0425 \u0411\u0415\u0417\u0421\u0415\u0420\u0412\u0415\u0420\u041d\u0418\u0425 \u0421\u0418\u0421\u0422\u0415\u041c \u041c\u0415\u0422\u0410\u041c\u041e\u0420\u0424\u0406\u0427\u041d\u041e\u0413\u041e \u0422\u0415\u0421\u0422\u0423\u0412\u0410\u041d\u041d\u042f<\/p>\n

\u00a0<\/strong>\u0421\u0442\u043e\u0440\u0456\u043d\u043a\u0438: 107-115. \u041d\u043e\u043c\u0435\u0440: \u2116<\/strong>1<\/strong>, 202<\/strong>2<\/strong> (<\/strong>305<\/strong>)\u00a0 \"\"<\/a><\/strong>
\n\u00a0<\/strong>\u0410\u0432\u0442\u043e\u0440\u0438:<\/strong>
\nYakiv YUSYN
\nhttps:\/\/orcid.org\/0000-0001-6971-3808<\/a>
\ne-mail: yusin.yakiv@gmail.com<\/a>
\nTetiana ZABOLOTNIA
\nhttps:\/\/orcid.org\/0000-0001-8570-7571<\/a>
\ne-mail: tetiana.zabolotnia@gmail.com<\/a>
\nNational Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”<\/p>\n

\u042e\u0421\u0418\u041d \u042f. \u041e., \u0417\u0410\u0411\u041e\u041b\u041e\u0422\u041d\u042f \u0422. \u041c.
\n\u041d\u0430\u0446\u0456\u043e\u043d\u0430\u043b\u044c\u043d\u0438\u0439 \u0442\u0435\u0445\u043d\u0456\u0447\u043d\u0438\u0439 \u0443\u043d\u0456\u0432\u0435\u0440\u0441\u0438\u0442\u0435\u0442 \u0423\u043a\u0440\u0430\u0457\u043d\u0438 \u00ab\u041a\u0438\u0457\u0432\u0441\u044c\u043a\u0438\u0439 \u043f\u043e\u043b\u0456\u0442\u0435\u0445\u043d\u0456\u0447\u043d\u0438\u0439 \u0456\u043d\u0441\u0442\u0438\u0442\u0443\u0442 \u0456\u043c. \u0406\u0433\u043e\u0440\u044f \u0421\u0456\u043a\u043e\u0440\u0441\u044c\u043a\u043e\u0433\u043e\u00bb
\nDOI:<\/strong> https:\/\/www.doi.org\/10.31891\/2307-5732-2022-305-1-107-115<\/a><\/p>\n

\u0410\u043d\u043e\u0442\u0430\u0446\u0456\u044f \u043c\u043e\u0432\u043e\u044e \u043e\u0440\u0438\u0433\u0456\u043d\u0430\u043b\u0443<\/strong><\/p>\n

\u00a0<\/strong>The task of quality assurance of software systems in IT is still an urgent problem, and due to the growing complexity of these systems is becoming increasingly difficult to use old methods of automated testing. One of the new methods of automated testing is metamorphic testing, which can be applied to systems of any complexity and which can be performed efficiently in the cloud. However, software for performing metamorphic tests in the cloud is still in the early stages of its development, due to the still low popularity of the method in the industry. So, the purpose of this work is to improve the software to perform metamorphic tests in the cloud by developing the corresponding design pattern to improve the expected results against the software code quality metrics. To achieve this objective, the new MTaaS design pattern has been developed that is based on the idea of metamorphic relation decomposition into individual parts together with automatic code generation of the relations’ and functions’ bodies. The combination of these two ideas allows the developer to concentrate only on the implementation of the logic of metamorphic relations, hiding from him all other details (such as creating serverless functions). To evaluate the developed design pattern, two software systems for metamorphic testing of the same software artifact were developed: one software system was implemented without the use of the MTaaS pattern, the other with its use. The following four code quality metrics were used in this evaluation: maintainability index, cyclomatic complexity, class coupling, lines of code. The analysis of evaluation results has demonstrated improvement of the class coupling and maintainability index metrics without worsening other metrics. Thus, the evaluation showed the effectiveness of using the developed design pattern during developing software systems for metamorphic testing based on serverless computing.
\nKeywords:<\/strong> metamorphic testing; cloud computing; serverless computing; design pattern.<\/p>\n

\u0420\u043e\u0437\u0448\u0438\u0440\u0435\u043d\u0430 \u0430\u043d\u043e\u0442\u0430\u0446\u0456\u044f \u0443\u043a\u0440\u0430\u0457\u043d\u0441\u044c\u043a\u043e\u044e\u00a0 \u043c\u043e\u0432\u043e\u044e<\/strong><\/p>\n

\u0417\u0430\u0434\u0430\u0447\u0430 \u0437\u0430\u0431\u0435\u0437\u043f\u0435\u0447\u0435\u043d\u043d\u044f \u044f\u043a\u043e\u0441\u0442\u0456 \u043f\u0440\u043e\u0433\u0440\u0430\u043c\u043d\u0438\u0445 \u0441\u0438\u0441\u0442\u0435\u043c \u0432 \u0406\u0422 \u0434\u043e\u0441\u0456 \u0437\u0430\u043b\u0438\u0448\u0430\u0454\u0442\u044c\u0441\u044f \u0430\u043a\u0442\u0443\u0430\u043b\u044c\u043d\u043e\u044e, \u043f\u0440\u0438 \u0447\u043e\u043c\u0443 \u0447\u0435\u0440\u0435\u0437 \u0437\u0440\u043e\u0441\u0442\u0430\u044e\u0447\u0443 \u0441\u043a\u043b\u0430\u0434\u043d\u0456\u0441\u0442\u044c \u0446\u0438\u0445 \u0441\u0438\u0441\u0442\u0435\u043c \u0432\u0441\u0435 \u0441\u043a\u043b\u0430\u0434\u043d\u0456\u0448\u0435 \u0441\u0442\u0430\u0454 \u0437\u0430\u0441\u0442\u043e\u0441\u0443\u0432\u0430\u043d\u043d\u044f \u0441\u0442\u0430\u0440\u0438\u0445 \u043c\u0435\u0442\u043e\u0434\u0456\u0432 \u0430\u0432\u0442\u043e\u043c\u0430\u0442\u0438\u0437\u043e\u0432\u0430\u043d\u043e\u0433\u043e \u0442\u0435\u0441\u0442\u0443\u0432\u0430\u043d\u043d\u044f. \u041e\u0434\u043d\u0438\u043c \u0456\u0437 \u043d\u043e\u0432\u0438\u0445 \u043c\u0435\u0442\u043e\u0434\u0456\u0432 \u0430\u0432\u0442\u043e\u043c\u0430\u0442\u0438\u0437\u043e\u0432\u0430\u043d\u043e\u0433\u043e \u0442\u0435\u0441\u0442\u0443\u0432\u0430\u043d\u043d\u044f \u0454 \u043c\u0435\u0442\u0430\u043c\u043e\u0440\u0444\u0456\u0447\u043d\u0435 \u0442\u0435\u0441\u0442\u0443\u0432\u0430\u043d\u043d\u044f, \u044f\u043a\u0435 \u043c\u043e\u0436\u0435 \u0431\u0443\u0442\u0438 \u0437\u0430\u0441\u0442\u043e\u0441\u043e\u0432\u0430\u043d\u0438\u043c \u0434\u043e \u0441\u0438\u0441\u0442\u0435\u043c \u0431\u0443\u0434\u044c-\u044f\u043a\u043e\u0457 \u0441\u043a\u043b\u0430\u0434\u043d\u043e\u0441\u0442\u0456 \u0442\u0430 \u044f\u043a\u0435 \u043c\u043e\u0436\u0435 \u0435\u0444\u0435\u043a\u0442\u0438\u0432\u043d\u043e \u0432\u0438\u043a\u043e\u043d\u0443\u0432\u0430\u0442\u0438\u0441\u044c \u0443 \u0445\u043c\u0430\u0440\u0456. \u041f\u0440\u043e\u0442\u0435, \u043f\u0440\u043e\u0433\u0440\u0430\u043c\u043d\u0435 \u0437\u0430\u0431\u0435\u0437\u043f\u0435\u0447\u0435\u043d\u043d\u044f \u0434\u043b\u044f \u0432\u0438\u043a\u043e\u043d\u0430\u043d\u043d\u044f \u043c\u0435\u0442\u0430\u043c\u043e\u0440\u0444\u0456\u0447\u043d\u0438\u0445 \u0442\u0435\u0441\u0442\u0456\u0432 \u0443 \u0445\u043c\u0430\u0440\u0456 \u0434\u043e\u0441\u0456 \u0437\u043d\u0430\u0445\u043e\u0434\u0438\u0442\u044c\u0441\u044f \u0443 \u043f\u043e\u0447\u0430\u0442\u043a\u043e\u0432\u0456\u0439 \u0444\u0430\u0437\u0456 \u0441\u0432\u043e\u0433\u043e \u0440\u043e\u0437\u0432\u0438\u0442\u043a\u0443, \u0449\u043e \u0432\u0438\u043a\u043b\u0438\u043a\u0430\u043d\u043e \u043f\u043e\u043a\u0438 \u0449\u043e \u043d\u0435\u0432\u0435\u043b\u0438\u043a\u043e\u044e \u043f\u043e\u043f\u0443\u043b\u044f\u0440\u043d\u0456\u0441\u0442\u044e \u043c\u0435\u0442\u043e\u0434\u0443 \u0432 \u0456\u043d\u0434\u0443\u0441\u0442\u0440\u0456\u0457.\u00a0 \u0412\u0456\u0434\u043f\u043e\u0432\u0456\u0434\u043d\u043e, \u043c\u0435\u0442\u043e\u044e \u0434\u0430\u043d\u043e\u0457 \u0440\u043e\u0431\u043e\u0442\u0438 \u0454 \u0443\u0434\u043e\u0441\u043a\u043e\u043d\u0430\u043b\u0435\u043d\u043d\u044f \u043f\u0440\u043e\u0433\u0440\u0430\u043c\u043d\u043e\u0433\u043e \u0437\u0430\u0431\u0435\u0437\u043f\u0435\u0447\u0435\u043d\u043d\u044f \u0434\u043b\u044f \u0432\u0438\u043a\u043e\u043d\u0430\u043d\u043d\u044f \u043c\u0435\u0442\u0430\u043c\u043e\u0440\u0444\u0456\u0447\u043d\u0438\u0445 \u0442\u0435\u0441\u0442\u0456\u0432 \u0443 \u0445\u043c\u0430\u0440\u0456 \u0448\u043b\u044f\u0445\u043e\u043c \u0440\u043e\u0437\u0440\u043e\u0431\u043b\u0435\u043d\u043d\u044f \u0432\u0456\u0434\u043f\u043e\u0432\u0456\u0434\u043d\u043e\u0433\u043e \u0448\u0430\u0431\u043b\u043e\u043d\u0443 \u043f\u0440\u043e\u0454\u043a\u0442\u0443\u0432\u0430\u043d\u043d\u044f, \u0449\u043e \u043f\u043e\u043a\u0440\u0430\u0449\u0438\u0442\u044c \u043e\u0442\u0440\u0438\u043c\u0443\u0432\u0430\u043d\u0456 \u0440\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u0438 \u0437\u0430 \u043c\u0435\u0442\u0440\u0438\u043a\u0430\u043c\u0438 \u044f\u043a\u043e\u0441\u0442\u0456 \u043f\u0440\u043e\u0433\u0440\u0430\u043c\u043d\u043e\u0433\u043e \u043a\u043e\u0434\u0443. \u0414\u043b\u044f \u0434\u043e\u0441\u044f\u0433\u043d\u0435\u043d\u043d\u044f \u043f\u043e\u0441\u0442\u0430\u0432\u043b\u0435\u043d\u043e\u0457 \u043c\u0435\u0442\u0438 \u0440\u043e\u0437\u0440\u043e\u0431\u043b\u0435\u043d\u043e \u043d\u043e\u0432\u0438\u0439 \u0448\u0430\u0431\u043b\u043e\u043d \u043f\u0440\u043e\u0454\u043a\u0442\u0443\u0432\u0430\u043d\u043d\u044f MTaaS, \u0432 \u043e\u0441\u043d\u043e\u0432\u0443 \u044f\u043a\u043e\u0433\u043e \u043f\u043e\u043a\u043b\u0430\u0434\u0435\u043d\u043e \u0456\u0434\u0435\u0457 \u0434\u0435\u043a\u043e\u043c\u043f\u043e\u0437\u0438\u0446\u0456\u0457 \u043c\u0435\u0442\u0430\u043c\u043e\u0440\u0444\u0456\u0447\u043d\u043e\u0433\u043e \u0437\u0432\u2019\u044f\u0437\u043a\u0443 \u043d\u0430 \u043e\u043a\u0440\u0435\u043c\u0456 \u0441\u043a\u043b\u0430\u0434\u043e\u0432\u0456 \u0440\u0430\u0437\u043e\u043c \u0437 \u0430\u0432\u0442\u043e\u043c\u0430\u0442\u0438\u0447\u043d\u043e\u044e \u043a\u043e\u0434\u043e\u0433\u0435\u043d\u0435\u0440\u0430\u0446\u0456\u0454\u044e \u0442\u0456\u043b \u0437\u0432\u2019\u044f\u0437\u043a\u0456\u0432 \u0442\u0430 \u0444\u0443\u043d\u043a\u0446\u0456\u0439. \u041f\u043e\u0454\u0434\u043d\u0430\u043d\u043d\u044f \u0446\u0438\u0445 \u0434\u0432\u043e\u0445 \u0456\u0434\u0435\u0439 \u0434\u043e\u0437\u0432\u043e\u043b\u044f\u0454 \u0440\u043e\u0437\u0440\u043e\u0431\u043d\u0438\u043a\u0443 \u0437\u043e\u0441\u0435\u0440\u0435\u0434\u0438\u0442\u0438\u0441\u044c \u0442\u0456\u043b\u044c\u043a\u0438 \u043d\u0430 \u043d\u0430\u043f\u0438\u0441\u0430\u043d\u0456 \u043b\u043e\u0433\u0456\u043a\u0438 \u043c\u0435\u0442\u0430\u043c\u043e\u0440\u0444\u0456\u0447\u043d\u0438\u0445 \u0437\u0432\u2019\u044f\u0437\u043a\u0456\u0432, \u043f\u0440\u0438\u0445\u043e\u0432\u0443\u044e\u0447\u0438 \u0432\u0456\u0434 \u043d\u044c\u043e\u0433\u043e \u0432\u0441\u0456 \u0456\u043d\u0448\u0456 \u0434\u0435\u0442\u0430\u043b\u0456 (\u0442\u0430\u043a\u0456 \u044f\u043a \u0441\u0442\u0432\u043e\u0440\u0435\u043d\u043d\u044f \u0431\u0435\u0437\u0441\u0435\u0440\u0432\u0435\u0440\u043d\u0438\u0445 \u0444\u0443\u043d\u043a\u0446\u0456\u0439). \u0414\u043b\u044f \u043e\u0446\u0456\u043d\u044e\u0432\u0430\u043d\u043d\u044f \u0440\u043e\u0437\u0440\u043e\u0431\u043b\u0435\u043d\u043e\u0433\u043e \u0448\u0430\u0431\u043b\u043e\u043d\u0443 \u043f\u0440\u043e\u0454\u043a\u0442\u0443\u0432\u0430\u043d\u043d\u044f \u0440\u043e\u0437\u0440\u043e\u0431\u043b\u0435\u043d\u043e \u0434\u0432\u0456 \u043f\u0440\u043e\u0433\u0440\u0430\u043c\u043d\u0456 \u0441\u0438\u0441\u0442\u0435\u043c\u0438 \u043c\u0435\u0442\u0430\u043c\u043e\u0440\u0444\u0456\u0447\u043d\u043e\u0433\u043e \u0442\u0435\u0441\u0442\u0443\u0432\u0430\u043d\u043d\u044f \u043e\u0434\u043d\u043e\u0433\u043e \u0456 \u0442\u043e\u0433\u043e \u0441\u0430\u043c\u043e\u0433\u043e \u043f\u0440\u043e\u0433\u0440\u0430\u043c\u043d\u043e\u0433\u043e \u0430\u0440\u0442\u0435\u0444\u0430\u043a\u0442\u0443: \u043e\u0434\u043d\u0430 \u043f\u0440\u043e\u0433\u0440\u0430\u043c\u043d\u0430 \u0441\u0438\u0441\u0442\u0435\u043c\u0430 \u0440\u0435\u0430\u043b\u0456\u0437\u043e\u0432\u0430\u043d\u0430 \u0431\u0435\u0437 \u0432\u0438\u043a\u043e\u0440\u0438\u0441\u0442\u0430\u043d\u043d\u044f \u0448\u0430\u0431\u043b\u043e\u043d\u0443 MTaaS, \u0456\u043d\u0448\u0430 \u0437 \u0439\u043e\u0433\u043e \u0432\u0438\u043a\u043e\u0440\u0438\u0441\u0442\u0430\u043d\u043d\u044f\u043c. \u041f\u0440\u0438 \u043e\u0446\u0456\u043d\u044e\u0432\u0430\u043d\u0456 \u0432\u0438\u043a\u043e\u0440\u0438\u0441\u0442\u0430\u043d\u043e \u043d\u0430\u0441\u0442\u0443\u043f\u043d\u0456 \u0447\u043e\u0442\u0438\u0440\u0438 \u043c\u0435\u0442\u0440\u0438\u043a\u0438 \u044f\u043a\u043e\u0441\u0442\u0456 \u043f\u0440\u043e\u0433\u0440\u0430\u043c\u043d\u043e\u0433\u043e \u043a\u043e\u0434\u0443: \u0456\u043d\u0434\u0435\u043a\u0441 \u043f\u0456\u0434\u0442\u0440\u0438\u043c\u0443\u0432\u0430\u043d\u043e\u0441\u0442\u0456, \u0446\u0438\u043a\u043b\u043e\u043c\u0430\u0442\u0438\u0447\u043d\u0430 \u0441\u043a\u043b\u0430\u0434\u043d\u0456\u0441\u0442\u044c, \u0437\u0432\u2019\u044f\u0437\u043d\u0456\u0441\u0442\u044c \u043a\u043b\u0430\u0441\u0456\u0432, \u043a\u0456\u043b\u044c\u043a\u0456\u0441\u0442\u044c \u0440\u044f\u0434\u043a\u0456\u0432 \u043a\u043e\u0434\u0443. \u0410\u043d\u0430\u043b\u0456\u0437 \u043e\u0442\u0440\u0438\u043c\u0430\u043d\u0438\u0445 \u0440\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u0456\u0432 \u043f\u043e\u043a\u0430\u0437\u0430\u0432 \u043f\u043e\u043a\u0440\u0430\u0449\u0435\u043d\u043d\u044f \u043c\u0435\u0442\u0440\u0438\u043a \u0456\u043d\u0434\u0435\u043a\u0441\u0430 \u043f\u0456\u0434\u0442\u0440\u0438\u043c\u0443\u0432\u0430\u043d\u043e\u0441\u0442\u0456 \u0442\u0430 \u0437\u0432\u2019\u044f\u0437\u043d\u043e\u0441\u0442\u0456 \u043a\u043b\u0430\u0441\u0456\u0432, \u043f\u0440\u0438 \u0432\u0456\u0434\u0441\u0443\u0442\u043d\u043e\u0441\u0442\u0456 \u043f\u043e\u0433\u0456\u0440\u0448\u0435\u043d\u043d\u044f \u0456\u043d\u0448\u0438\u0445 \u0434\u0432\u043e\u0445. \u0422\u0430\u043a\u0438\u043c \u0447\u0438\u043d\u043e\u043c, \u043f\u0440\u043e\u0432\u0435\u0434\u0435\u043d\u0435 \u043e\u0446\u0456\u043d\u044e\u0432\u0430\u043d\u043d\u044f \u043f\u043e\u043a\u0430\u0437\u0430\u043b\u043e \u0435\u0444\u0435\u043a\u0442\u0438\u0432\u043d\u0456\u0441\u0442\u044c \u0432\u0438\u043a\u043e\u0440\u0438\u0441\u0442\u0430\u043d\u043d\u044f \u0440\u043e\u0437\u0440\u043e\u0431\u043b\u0435\u043d\u043e\u0433\u043e \u0448\u0430\u0431\u043b\u043e\u043d\u0443 \u043f\u0440\u043e\u0454\u043a\u0442\u0443\u0432\u0430\u043d\u043d\u044f \u043f\u0440\u0438 \u0440\u043e\u0437\u0440\u043e\u0431\u043b\u0435\u043d\u0456 \u043f\u0440\u043e\u0433\u0440\u0430\u043c\u043d\u0438\u0445 \u0441\u0438\u0441\u0442\u0435\u043c \u043c\u0435\u0442\u0430\u043c\u043e\u0440\u0444\u0456\u0447\u043d\u043e\u0433\u043e \u0442\u0435\u0441\u0442\u0443\u0432\u0430\u043d\u043d\u044f \u043d\u0430 \u043e\u0441\u043d\u043e\u0432\u0456 \u0431\u0435\u0437\u0441\u0435\u0440\u0432\u0435\u0440\u043d\u0438\u0445 \u043e\u0431\u0447\u0438\u0441\u043b\u0435\u043d\u044c.
\n\u041a\u043b\u044e\u0447\u043e\u0432\u0456 \u0441\u043b\u043e\u0432\u0430:<\/strong> \u043c\u0435\u0442\u0430\u043c\u043e\u0440\u0444\u0456\u0447\u043d\u0435 \u0442\u0435\u0441\u0442\u0443\u0432\u0430\u043d\u043d\u044f; \u0445\u043c\u0430\u0440\u043d\u0456 \u043e\u0431\u0447\u0438\u0441\u043b\u0435\u043d\u043d\u044f; \u0431\u0435\u0437\u0441\u0435\u0440\u0432\u0435\u0440\u043d\u0456 \u043e\u0431\u0447\u0438\u0441\u043b\u0435\u043d\u043d\u044f; \u0448\u0430\u0431\u043b\u043e\u043d \u043f\u0440\u043e\u0454\u043a\u0442\u0443\u0432\u0430\u043d\u043d\u044f.<\/p>\n

References<\/strong><\/p>\n

    \n
  1. Capgemini, Sogeti, Microfocus, \u201cWorld Quality Report 2021\u201d. URL: https:\/\/www.capgemini.com\/research\/world-quality-report-wqr-2021-22\/.<\/li>\n
  2. Weyuker E., \u201cThe oracle assumption of program testing,\u201d in Proc. of 13th International Conference on System Sciences, pp. 44-49 (1980).<\/li>\n
  3. Barr T., Harman M., McMinn P., Shahbaz M., Yoo S., \u201cThe oracle problem in software testing: a survey,\u201d IEEE Transactions on Software Engineering Vol. 41 (5), pp. 507\u2013525 (2015). https:\/\/doi.org\/10.1109\/TSE.2014.2372785.<\/li>\n
  4. Chen T.Y., Cheung S.C., Yiu S.M., \u201cMetamorphic testing: a new approach for generating next test cases\u201d Technical Report HKUST-CS98-01, Department of Computer Science, Hong Kong University of Science and Technology, Hong Kong (1998).<\/li>\n
  5. Zhou Z.Q., Zhang S., Hagenbuchner M., Tse T.H., Kuo F.-C., Chen T.Y., \u201cAutomated functional testing of online search services\u201d Software Testing, Veri\ufb01cation and Reliability Journal Vol. 22 (4), pp. 221\u2013243 (2012). https:\/\/doi.org\/10.1002\/stvr.437.<\/li>\n
  6. Zhou Z.G., Tse T.H., Kuo F.-C., Chen T.Y. \u201cAutomated functional testing of web search engines in the absence of an oracle\u201d Technical Report TR-2007-06, Department of Computer Science, The University of Hong Kong, Hong Kong (2007).<\/li>\n
  7. Zhou Q., Xiang S., Chen T.Y. \u201cMetamorphic testing for software quality assessment: A study of search engines\u201d IEEE Transactions on Software Engineering Vol. 42 (3), pp. 264\u2013284 (2016). https:\/\/doi.org\/10.1109\/TSE.2015.2478001.<\/li>\n
  8. Tao Q., Wu W., Zhao C., Shen W. \u201cAn automatic testing approach for compiler based on metamorphic testing technique\u201d in Proc. of 17th Asia Paci\ufb01c Software Engineering Conference (APSEC), pp. 270\u2013279 (2010). https:\/\/doi.org\/10.1109\/APSEC.2010.39.<\/li>\n
  9. Le V., Afshari M., Su Z. \u201cCompiler validation via equivalence modulo inputs\u201d in Proc. of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation, pp. 216\u2013226 (2014). https:\/\/doi.org\/10.1145\/2666356.2594334.<\/li>\n
  10. Kuo F.-C., Liu S., Chen T.Y. \u201cTesting a binary space partitioning algorithm with metamorphic testing\u201d in Proc. of the 2011 ACM Symposium on Applied Computing, pp. 1482\u20131489 (2011). http:\/\/dx.doi.org\/10.1145\/1982185.1982502.<\/li>\n
  11. Jameel T., Mengxiang L., Liu C. \u201cTest oracles based on metamorphic relations for image processing applications\u201d in Proc. of 16th IEEE\/ACIS International Conference on Software Engineering, Arti\ufb01cial Intelligence, Networking and Parallel\/Distributed Computing (SNPD), pp. 1\u20136 (2015). https:\/\/doi.org\/10.1109\/SNPD.2015.7176238.<\/li>\n
  12. Pullum L.L., Ozmen O. \u201cEarly results from metamorphic testing of epidemiological models\u201d in Proc. of ASE\/IEEE International Conference on BioMedical Computing (BioMedCom), pp. 62\u201367 (2012). https:\/\/doi.org\/10.1109\/BioMedCom.2012.17.<\/li>\n
  13. Ramanathan A., Steed C.A., Pullum L.L. \u201cVeri\ufb01cation of compartmental epidemiological models using metamorphic testing, model checking and visual analytics\u201d in Proc. of ASE\/IEEE International Conference on BioMedical Computing (BioMedCom), pp. 68\u201373 (2012). https:\/\/doi.org\/10.1109\/BioMedCom.2012.18.<\/li>\n
  14. Segura , Fraser G., Sanchez A.B., Ruiz-Cortes A. \u201cA survey on metamorphic testing\u201d IEEE Transactions on Software Engineering Vol. 42 (9), pp. 805-824 (2016). https:\/\/doi.org\/10.1109\/TSE.2016.2532875.<\/li>\n
  15. Chen T.Y., Kuo F.-C., Liu H., Poon P.-L., Towey D., Tse T.H., Zhou Z.Q. \u201cMetamorphic testing: A review of challenges and opportunities\u201d ACM Computing Surveys Vol. 51 (1), pp. 4:1-4:27 (2018). https:\/\/doi.org\/10.1145\/3143561.<\/li>\n<\/ol>\n
      \n
    1. Troup M., Yang A., Kamali A.H., Giannoulatou E., Chen T.Y., Joshua W. K. \u201cA cloud-based framework for applying metamorphic testing to a bioinformatics pipeline\u201d In Proc. of the 1st International Workshop on Metamorphic Testing, pp. 33\u201336 (2016). https:\/\/doi.org\/10.1109\/MET.2016.014.<\/li>\n
    2. Yusyn Y., Zabolotnia T. \u201cMetamorphic Testing and Serverless Computing: A Basic Architecture\u201d preprint, [Online]. Available: https:\/\/yakivyusin.github.io\/preprints\/Serverless.pdf.<\/li>\n
    3. Gamma E., Helm R., Johnson R., Vlissides J. \u201cDesign Patterns: Elements of Reusable Object-Oriented Software\u201d Addison-Wesley (1994).<\/li>\n
    4. Microsoft, \u201cSource Generators\u201d. URL: https:\/\/docs.microsoft.com\/en-us\/dotnet\/csharp\/roslyn-sdk\/source-generators-overview.<\/li>\n
    5. YAML Language Development Team, \u201cYAML Ain\u2019t Markup Language (YAML™) version 1.2\u201d. Available: https:\/\/yaml.org\/spec\/1.2.2\/.<\/li>\n
    6. Microsoft, \u201cCode metrics \u2013 Maintainability index range and meaning\u201d URL: https:\/\/docs.microsoft.com\/en-us\/visualstudio\/code-quality\/code-metrics-maintainability-index-range-and-meaning.<\/li>\n
    7. Microsoft, \u201cCode metrics \u2013 Cyclomatic complexity\u201d URL: https:\/\/docs.microsoft.com\/en-us\/visualstudio\/code-quality\/code-metrics-cyclomatic-complexity.<\/li>\n
    8. Microsoft, \u201cCode metrics \u2013 Class coupling\u201d URL: https:\/\/docs.microsoft.com\/en-us\/visualstudio\/code-quality\/code-metrics-class-coupling.<\/li>\n<\/ol>\n

       <\/p>\n","protected":false},"excerpt":{"rendered":"","protected":false},"author":3,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[62],"tags":[],"_links":{"self":[{"href":"https:\/\/journals.khnu.km.ua\/vestnik\/index.php?rest_route=\/wp\/v2\/posts\/12134"}],"collection":[{"href":"https:\/\/journals.khnu.km.ua\/vestnik\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/journals.khnu.km.ua\/vestnik\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/journals.khnu.km.ua\/vestnik\/index.php?rest_route=\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/journals.khnu.km.ua\/vestnik\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=12134"}],"version-history":[{"count":4,"href":"https:\/\/journals.khnu.km.ua\/vestnik\/index.php?rest_route=\/wp\/v2\/posts\/12134\/revisions"}],"predecessor-version":[{"id":12208,"href":"https:\/\/journals.khnu.km.ua\/vestnik\/index.php?rest_route=\/wp\/v2\/posts\/12134\/revisions\/12208"}],"wp:attachment":[{"href":"https:\/\/journals.khnu.km.ua\/vestnik\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=12134"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/journals.khnu.km.ua\/vestnik\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=12134"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/journals.khnu.km.ua\/vestnik\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=12134"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}