{"id":3912,"date":"2018-02-22T21:28:07","date_gmt":"2018-02-23T05:28:07","guid":{"rendered":"https:\/\/upon2020.com\/blog\/?p=3912"},"modified":"2018-02-22T21:28:07","modified_gmt":"2018-02-23T05:28:07","slug":"automated-testing-for-the-indie-web","status":"publish","type":"post","link":"https:\/\/upon2020.com\/blog\/2018\/02\/automated-testing-for-the-indie-web\/","title":{"rendered":"Automated Testing for the Indie Web?"},"content":{"rendered":"<p>(This post is mostly for <a href=\"https:\/\/indieweb.org\/\">Indie Web<\/a> geeks. Feel free to ignore)<\/p>\n<h3>The problem<\/h3>\n<ul>\n<li>We spend too much time debugging why things didn&#8217;t work as expected<\/li>\n<li>Not so much Gen 1 people running their own, custom-developed software, but later-gen people who try to, say, make WordPress work with a previously unseen combination of plugins and themes<\/li>\n<\/ul>\n<h3>Proposed solution: systematic, preferably automated, testing<\/h3>\n<p>(Note: the remainder focuses on microformats only. Of course there&#8217;s much more to test, but we got to start somewhere&#8230;)<\/p>\n<h4>Example test case 1<\/h4>\n<ul>\n<li>Given a new installation of WordPress, with N installed themes.<\/li>\n<li>Given the URL to the &#8220;installation succeeded&#8221; default WordPress post that comes with<br \/>\nall new WordPress installations (lets call it WP_POST_URL).<\/li>\n<li>For each of the installed themes:\n<ul>\n<li>The microformat content of the page at URL WP_POST_URL, as parsed by mf2py,<br \/>\nmust be the same.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>(This may be a bit more complicated &#8212; e.g. some theme might show only one post, while<br \/>\nothers have references to other posts etc &#8212; but as I put it, it&#8217;s good enough to get a<br \/>\ndiscussion going)<\/p>\n<h5>Example test case 2<\/h5>\n<ul>\n<li>WordPress has a new version, but no other changes.<\/li>\n<li>Verify that the microformats on my blog are still interpreted the same way after the<br \/>\nupgrade.<\/li>\n<\/ul>\n<h3>Considerations for possible implementations<\/h3>\n<ul>\n<li>QA is very time consuming, and unless you get paid (and often not even then), nobody<br \/>\nlikes to do it on an ongoing basis.<\/li>\n<li>I&#8217;m biased because I use a self-written utility called <a href=\"https:\/\/ubos.net\/docs\/developers\/app-test.html\">webapptest<\/a> to test that web<br \/>\napplications on UBOS still run after various OS\/app\/middleware upgrades. It could be<br \/>\nadapted to our purpose here. Here&#8217;s how it would work:<\/p>\n<ul>\n<li>The script instantiates a new WordPress site temporarily in a Linux container,<br \/>\nwith a configurable set of plugins and themes.<\/li>\n<li>The script restores previously saved WordPress site data (database, uploaded files)<\/li>\n<li>The script runs mf2py on a set of URLs pointing to various pages of the WordPress<br \/>\ninstall<\/li>\n<li>The script compares the output with a previously saved baseline.<\/li>\n<li>If there is a discrepancy, emit an error<\/li>\n<li>The Linux container is torn down.<\/li>\n<\/ul>\n<\/li>\n<li>The work involved would be:\n<ul>\n<li>Create one\/several WordPress sites with &#8220;interesting&#8221; content, and save their<br \/>\ndatabases\/uploaded files.<\/li>\n<li>Identify &#8220;interesting&#8221; URLs for each<\/li>\n<li>Run mf2py against the interesting URLs, and save the output<\/li>\n<li>Augment webapptest utility to load WordPress backups once it has installed<br \/>\nthe sites temporarily in a container (it knows how to do that today), and run<br \/>\nmf2py and some kind of JSON diff instead of the default webapptest scripts.<\/li>\n<li>Create an overall script that iterates through the test cases.<\/li>\n<li>&#8230; and then run this on a regular basis.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>This would apply similarly to tools other than WordPress, of course.<\/p>\n<p>Thoughts?<\/p>\n","protected":false},"excerpt":{"rendered":"<p>(This post is mostly for Indie Web geeks. Feel free to ignore) The problem We spend too much time debugging why things didn&#8217;t work as expected Not so much Gen 1 people running their own, custom-developed software, but later-gen people who try to, say, make Wordpress work with a previously unseen combination of plugins and&hellip;<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"webmentions_disabled":false,"footnotes":""},"categories":[301,66],"tags":[550],"class_list":["post-3912","post","type-post","status-publish","format-standard","hentry","category-indie-web","category-technical","tag-qa","kind-article"],"kind":false,"_links":{"self":[{"href":"https:\/\/upon2020.com\/blog\/wp-json\/wp\/v2\/posts\/3912","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/upon2020.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/upon2020.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/upon2020.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/upon2020.com\/blog\/wp-json\/wp\/v2\/comments?post=3912"}],"version-history":[{"count":5,"href":"https:\/\/upon2020.com\/blog\/wp-json\/wp\/v2\/posts\/3912\/revisions"}],"predecessor-version":[{"id":3917,"href":"https:\/\/upon2020.com\/blog\/wp-json\/wp\/v2\/posts\/3912\/revisions\/3917"}],"wp:attachment":[{"href":"https:\/\/upon2020.com\/blog\/wp-json\/wp\/v2\/media?parent=3912"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/upon2020.com\/blog\/wp-json\/wp\/v2\/categories?post=3912"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/upon2020.com\/blog\/wp-json\/wp\/v2\/tags?post=3912"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}