.1 200 OK date: Wed, 01 Jul 2026 23:31:35 GMT server: mw-web.codfw.main-77cfb8ddf-r4nvt x-content-type-options: nosniff content-language: en accept-ch: reporting-endpoints: csp-report-to-endpoint='/w/api.php?action=cspreport&format=json'; content-security-policy: script-src 'unsafe-eval' blob: 'self' meta.wikimedia.org *.wikimedia.org *.wikipedia.org *.wikinews.org *.wiktionary.org *.wikibooks.org *.wikiversity.org *.wikisource.org wikisource.org *.wikiquote.org *.wikidata.org *.wikifunctions.org *.wikivoyage.org *.mediawiki.org mediawiki.org wikimedia.org *.wmflabs.org *.wmcloud.org *.toolforge.org wss://*.toolforge.org *.jsdelivr.net unpkg.com cdnjs.cloudflare.com raw.githubusercontent.com *.github.com code.jquery.com cdn.mathjax.org use.typekit.net fonts.cdnfonts.com use.fontawesome.com i.ytimg.com rsms.me doi.org localhost https://localhost:* http://localhost:* wss://localhost:* ws://localhost:* *.google.com *.gstatic.com *.googleapis.com *.translate.yandex.net yastatic.net ya.ru radically.github.io cdn.sammdot.ca cdn.fontshare.com viaf.org publicai-proxy.alaexis.workers.dev iiif.archive.org api.flickr.com live.staticflickr.com api.anthropic.com api.openai.com api.publicai.co catalogo.pusc.it parsifal.urbe.it opac.sbn.it overpass-api.de api.openrouteservice.org archive.org *.openstreetmap.org *.waymarkedtrails.org *.thunderforest.com registry.ipe.wiki analytics.ipe.wiki qlever.dev app.goacoustic.com wikipedia-archive.ourworldindata.org api.inaturalist.org inaturalist-open-data.s3.amazonaws.com validator.w3.org db.onlinewebfonts.com fontlibrary.org 'unsafe-inline' auth.wikimedia.org; default-src 'self' data: blob: upload.wikimedia.org https://commons.wikimedia.org meta.wikimedia.org *.wikimedia.org *.wikipedia.org *.wikinews.org *.wiktionary.org *.wikibooks.org *.wikiversity.org *.wikisource.org wikisource.org *.wikiquote.org *.wikidata.org *.wikifunctions.org *.wikivoyage.org *.mediawiki.org mediawiki.org wikimedia.org *.wmflabs.org *.wmcloud.org *.toolforge.org wss://*.toolforge.org *.jsdelivr.net unpkg.com cdnjs.cloudflare.com raw.githubusercontent.com *.github.com code.jquery.com cdn.mathjax.org use.typekit.net fonts.cdnfonts.com use.fontawesome.com i.ytimg.com rsms.me doi.org localhost https://localhost:* http://localhost:* wss://localhost:* ws://localhost:* *.google.com *.gstatic.com *.googleapis.com *.translate.yandex.net yastatic.net ya.ru radically.github.io cdn.sammdot.ca cdn.fontshare.com viaf.org publicai-proxy.alaexis.workers.dev iiif.archive.org api.flickr.com live.staticflickr.com api.anthropic.com api.openai.com api.publicai.co catalogo.pusc.it parsifal.urbe.it opac.sbn.it overpass-api.de api.openrouteservice.org archive.org *.openstreetmap.org *.waymarkedtrails.org *.thunderforest.com registry.ipe.wiki analytics.ipe.wiki qlever.dev app.goacoustic.com wikipedia-archive.ourworldindata.org api.inaturalist.org inaturalist-open-data.s3.amazonaws.com validator.w3.org db.onlinewebfonts.com fontlibrary.org en.wikibooks.org en.wikinews.org en.wikiquote.org en.wikisource.org en.wikiversity.org en.wikivoyage.org en.wiktionary.org www.mediawiki.org commons.wikimedia.org foundation.wikimedia.org incubator.wikimedia.org species.wikimedia.org wikimania.wikimedia.org www.wikidata.org www.wikifunctions.org auth.wikimedia.org; style-src 'self' data: blob: upload.wikimedia.org https://commons.wikimedia.org meta.wikimedia.org *.wikimedia.org *.wikipedia.org *.wikinews.org *.wiktionary.org *.wikibooks.org *.wikiversity.org *.wikisource.org wikisource.org *.wikiquote.org *.wikidata.org *.wikifunctions.org *.wikivoyage.org *.mediawiki.org mediawiki.org wikimedia.org *.wmflabs.org *.wmcloud.org *.toolforge.org wss://*.toolforge.org *.jsdelivr.net unpkg.com cdnjs.cloudflare.com raw.githubusercontent.com *.github.com code.jquery.com cdn.mathjax.org use.typekit.net fonts.cdnfonts.com use.fontawesome.com i.ytimg.com rsms.me doi.org localhost https://localhost:* http://localhost:* wss://localhost:* ws://localhost:* *.google.com *.gstatic.com *.googleapis.com *.translate.yandex.net yastatic.net ya.ru radically.github.io cdn.sammdot.ca cdn.fontshare.com viaf.org publicai-proxy.alaexis.workers.dev iiif.archive.org api.flickr.com live.staticflickr.com api.anthropic.com api.openai.com api.publicai.co catalogo.pusc.it parsifal.urbe.it opac.sbn.it overpass-api.de api.openrouteservice.org archive.org *.openstreetmap.org *.waymarkedtrails.org *.thunderforest.com registry.ipe.wiki analytics.ipe.wiki qlever.dev app.goacoustic.com wikipedia-archive.ourworldindata.org api.inaturalist.org inaturalist-open-data.s3.amazonaws.com validator.w3.org db.onlinewebfonts.com fontlibrary.org 'unsafe-inline'; object-src 'none'; report-uri /w/api.php?action=cspreport&format=json; report-to csp-report-to-endpoint last-modified: Wed, 01 Jul 2026 08:40:05 GMT content-type: text/html; charset=UTF-8 age: 48720 accept-ranges: bytes x-cache: cp2047 miss, cp2047 hit/2 x-cache-status: hit-front server-timing: cache;desc="hit-front", host;desc="cp2047" strict-transport-security: max-age=106384710; includeSubDomains; preload report-to: { "group": "wm_nel", "max_age": 604800, "endpoints": [{ "url": "https://intake-logging.wikimedia.org/v1/events?stream=w3c.reportingapi.network_error&schema_uri=/w3c/reportingapi/network_error/1.0.0" }] } nel: { "report_to": "wm_nel", "max_age": 604800, "failure_fraction": 0.05, "success_fraction": 0.0} set-cookie: WMF-Last-Access=02-Jul-2026;Path=/;HttpOnly;secure;Expires=Mon, 03 Aug 2026 12:00:00 GMT set-cookie: WMF-Last-Access-Global=02-Jul-2026;Path=/;Domain=.wikipedia.org;HttpOnly;secure;Expires=Mon, 03 Aug 2026 12:00:00 GMT set-cookie: WMF-DP=025;Path=/;HttpOnly;secure;Expires=Fri, 03 Jul 2026 00:00:00 GMT x-client-ip: 75.28.18.94 cache-control: private, s-maxage=0, max-age=0, must-revalidate, no-transform vary: Accept-Encoding,X-Subdomain,Cookie,Authorization,User-Agent set-cookie: GeoIP=US:TX:McKinney:33.20:-96.62:v4; Path=/; secure; Domain=.wikipedia.org set-cookie: NetworkProbeLimit=0.001;Path=/;Secure;SameSite=None;Max-Age=3600 set-cookie: WMF-Uniq=5oKnADSgGItYfXYrdqtlNgORAAAAAFvd6JgsIE2ClLP0qQjDeVDI9tGFF97r8Z7C;Domain=.wikipedia.org;Path=/;HttpOnly;secure;SameSite=None;Expires=Fri, 02 Jul 2027 00:00:00 GMT content-length: 106498 x-request-id: 8a85e255-bfc4-4189-9d87-1de8cdb20ac2 x-analytics: Metamorphic testing - Wikipedia Jump to content Main menu Main menu move to sidebar hide Navigation Main page Contents Current events Random article About Wikipedia Contact us Contribute Help Learn to edit Community portal Recent changes Upload file Special pages Search Search Appearance Donate Create account Log in Personal tools Donate Create account Log in Contents move to sidebar hide (Top) 1 See also 2 References 3 External links Toggle the table of contents Metamorphic testing 2 languages Italiano 中文 Edit links Article Talk English Read Edit View history Tools Tools move to sidebar hide Actions Read Edit View history General What links here Related changes Upload file Permanent link Page information Cite this page Get shortened URL Switch to legacy parser Print/export Download as PDF Printable version In other projects Wikidata item Appearance move to sidebar hide From Wikipedia, the free encyclopedia Software testing technique Metamorphic testing (MT) is a property-based software testing technique, which can be an effective approach for addressing the test oracle problem and test case generation problem. The test oracle problem is the difficulty of determining the expected outcomes of selected test cases or to determine whether the actual outputs agree with the expected outcomes. Metamorphic relations (MRs) are necessary properties of the intended functionality of the software, and must involve multiple executions of the software. Consider, for example, a program that implements sin x correct to 100 significant figures ; a metamorphic relation for sine functions is " sin ( π − x ) = sin x ". Thus, even though the expected value of sin x 1 for the source test case x 1 = 1.234 correct to the required accuracy is not known, a follow-up test case x 2 = π − 1.234 can be constructed. We can verify whether the actual outputs produced by the program under test from the source test case and the follow-up test case are consistent with the MR in question. Any inconsistency (after taking rounding errors into consideration) indicates a failure [ 1 ] : 31 of the program, caused by a fault [ 1 ] : 31 in the implementation. MRs are not limited to programs with numerical inputs or equality relations. As an example, when testing a booking website, a web search for accommodation in Sydney, Australia, returns 1,671 results; are the results of this search correct and complete? This is a test oracle problem. Based on a metamorphic relation, we may filter the price range or star rating and apply the search again; it should return a subset of the previous results. A violation of this expectation would similarly reveal a failure of the system. Metamorphic testing was invented by T.Y. Chen in a technical report [ 2 ] in 1998. Since then, more than 750 papers have been published by international researchers and practitioners, working further on the theory and practice of MT in real-life applications. [ 3 ] Some examples include web services , [ 4 ] computer graphics , [ 5 ] embedded systems , [ 6 ] simulation and modeling , [ 7 ] machine learning , [ 8 ] decision support , [ 9 ] bioinformatics , [ 10 ] components , [ 11 ] numerical analysis , [ 12 ] compilers , [ 13 ] and even quantum computing [ 14 ] The first major survey of MT was conducted in 2016. [ 15 ] It was followed by another major survey in 2018, [ 16 ] which highlights the challenges and opportunities and clarifies common misunderstandings. Although MT was initially proposed as a software verification technique, it was later developed into a paradigm that covers verification, validation , and other types of software quality assessment. [ 17 ] MT can be applied independently, and can also be combined with other static and dynamic software analysis techniques such as proving and debugging . [ 18 ] In August 2018, Google acquired GraphicsFuzz, a startup from Imperial College London , to apply metamorphic testing to graphics device drivers for Android smartphones . [ 19 ] [ 20 ] See also [ edit ] Mutation testing Property testing QuickCheck References [ edit ] 1 2 610.12-1990 - IEEE Standard Glossary of Software Engineering Terminology , IEEE, 1990, doi : 10.1109/IEEESTD.1990.101064 , ISBN 9781559370677 ↑ T.Y. Chen; S.C. Cheung; S.M. Yiu (1998), "Metamorphic testing: A new approach for generating next test cases", Technical Report HKUST-CS98-01 (PDF) , Department of Computer Science, The Hong Kong University of Science and Technology , Hong Kong, arXiv : 2002.12543 . ↑ "allintitle: "metamorphic testing" OR "metamorphic relation" " . Google Scholar . Retrieved 11 July 2025 . ↑ C. Castro-Cabrera and I. Medina-Bulo, "An approach to metamorphic testing for WS-BPEL compositions", in Proceedings of the International Conference on e-Business , IEEE (2011). ↑ R. Guderlei and J. Mayer, "Towards automatic testing of imaging software by means of random and metamorphic testing", International Journal of Software Engineering and Knowledge Engineering 17 (6): 757−781 (2007). ↑ T.H. Tse , S.S. Yau, W.K. Chan, H. Lu, and T.Y. Chen, "Testing context-sensitive middleware-based software applications", in Proceedings of the 28th Annual International Computer Software and Applications Conference ( COMPSAC '04 ), vol. 1, IEEE Computer Society , pp. 458−465 (2004). ↑ S. Segura, R.M. Hierons, D. Benavides, and A. Ruiz-Cortes, "Automated test data generation on the analyses of feature models: a metamorphic testing approach", in Proceedings of the 3rd International Conference on Software Testing, Verification, and Validation ( ICST '10 ), IEEE Computer Society , pp. 35−44 (2010). ↑ J. Ding, X.-H. Hu, and V. Gudivada, "A machine learning based framework for verification and validation of massive scale image data", IEEE Transactions on Big Data (2017). doi: 10.1109/TBDATA.2017.2680460. ↑ F.-C. Kuo, Z.Q. Zhou, J. Ma, and G. Zhang, "Metamorphic testing of decision support systems: A case study", IET Software 4 (4): 294−301 (2010). ↑ L.L. Pullum and O. Ozmen, "Early results from metamorphic testing of epidemiological models", in Proceedings of the 2012 ASE/IEEE International Conference on BioMedical Computing ( BioMedCom '12 ), IEEE Computer Society , pp. 62−67 (2012). ↑ S. Beydeda, "Self-metamorphic-testing components", in Proceedings of the 30th Annual International Computer Software and Applications Conference ( COMPSAC '06 ), vol. 1, IEEE Computer Society , pp. 265−272 (2006). ↑ C. Aruna and R.S.R. Prasad, "Metamorphic relations to improve the test accuracy of multi precision arithmetic software applications", in Proceedings of the 2014 International Conference on Advances in Computing, Communications and Informatics ( ICACCI '14 ), IEEE (2014). ↑ C. Lidbury, A. Lascu, N. Chong, and A.F. Donaldson, "Many-core compiler fuzzing", in Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation ( PLDI '15 ), ACM , pp. 65−76 (2015). ↑ M. Paltenghi; M. Pradel (2023), "MorphQ: Metamorphic Testing of the Qiskit Quantum Computing Platform", 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) , ICSE '23, pp. 2413– 2424, arXiv : 2206.01111 , doi : 10.1109/ICSE48619.2023.00202 . ↑ S. Segura, G. Fraser, A.B. Sanchez, and A. Ruiz-Cortes, "A survey on metamorphic testing", IEEE Transactions on Software Engineering 42 (9}: 805-824 (2016). ↑ T.Y. Chen, F.-C. Kuo, H. Liu, P.-L. Poon, D. Towey, T.H. Tse , and Z.Q. Zhou, "Metamorphic testing: A review of challenges and opportunities", ACM Computing Surveys 51 (1): 4:1-4:27 (2018). ↑ Z.Q. Zhou, S. Xiang, and T.Y. Chen, "Metamorphic testing for software quality assessment: A study of search engines", IEEE Transactions on Software Engineering 42 (3): 264-28s4 (2016). ↑ T.Y. Chen, T.H. Tse , and Z.Q. Zhou, "Semi-proving: an integrated method for program proving, testing, and debugging", IEEE Transactions on Software Engineering 37 (1): 109-125 (2011). ↑ "GraphicsFuzz is acquired by Google" . GraphicsFuzz . Mountain View, CA , US . 6 August 2018 . Retrieved 23 December 2020 . ↑ Frederic Lardinois (6 August 2018). "Google acquires GraphicsFuzz, a service that tests Android graphics drivers" . TechCrunch . San Francisco , CA , US . Retrieved 11 July 2025 . The company's founders started their work at the Department of Computing at Imperial College London and received funding support from the U.K. Engineering and Physical Sciences Research Council and the TETRACOM EU project. External links [ edit ] "Metamorphic testing: Introduction and applications" . ACM SIGSOFT Webinar . Sergio Segura; Zhi Quan Zhou (2018). ICSE 2018 technical briefing on metamorphic testing . International Conference on Software Engineering (ICSE 2018). doi : 10.5281/zenodo.1256230 . Zhi Quan (George) Zhou (9 April 2019). Automated testing of real-life self-driving systems and beyond: Keynote speech . IEEE International Conference on Artificial Intelligence Testing (AI Test 2019). San Francisco East Bay, CA, USA. doi : 10.5281/zenodo.2636979 . Retrieved from " https://en.wikipedia.org/w/index.php?title=Metamorphic_testing&oldid=1351539255 " Category : Software testing Hidden categories: Articles with short description Short description matches Wikidata This page was last edited on 28 April 2026, at 16:32  (UTC) . Page was rendered with Parsoid . Text is available under the Creative Commons Attribution-ShareAlike 4.0 License ; additional terms may apply. By using this site, you agree to the Terms of Use and Privacy Policy . Wikipedia® is a registered trademark of the Wikimedia Foundation, Inc. , a non-profit organization. Privacy policy About Wikipedia Disclaimers Contact Wikipedia Legal & safety contacts Code of Conduct Developers Statistics Cookie statement Mobile view Search Search Toggle the table of contents Metamorphic testing 2 languages Add topic