A/B Testing with the Rivista CMS

A republishing of a blog post I wrote in 2014 for the company blog.

The following is a republishing of a blog post I wrote in 2014 for the GTxcel company blog.

The GTxcel product team recently took a step back to analyze the web traffic patterns on the first website ever launched on the Rivista 5 responsive web design content management system.  My New Orleans Magazine (www.myneworleans.com) readers are enjoying magazine content on their phones and tablets as well as on their desktop and laptop computers.

Since the launch in early 2013, the number of visitors on the site increased by almost 40% compared with the same period in 2012.  Clearly the Rivista 5 responsive redesign has had a positive impact on overall traffic.  Another good sign is that over one third of all visits are from mobile devices.  While most metrics were improving, the bounce rate remained relatively high in comparison with other online publications.  Upon further investigation, we realized that the article pages were not providing any links to other content.  Aside from the navigation menu in the header, visitors were essentially at a dead-end.  Since roughly half of all visitors link directly to article pages from sites like Google or Facebook, it’s not surprising that there was a high bounce rate.

Screenshot of a popular article about Shelby Stanga

We decided to run some A/B testing experiments to see if adding article list modules to the page would increase reader engagement and reduce bounce rate.  We chose an article about Shelby Stanga because it was getting a lot of traffic, mostly from searches.  The article is near the top of Google search results.  It also had a greater than 90% bounce rate – one of the highest.

The Shelby Stanga article was the most popular page on myneworleans.com during the summer months as season 6 of “Axe Men” aired this year, and the History channel released the new show “The Legend of Shelby the Swamp Man” this summer.  The article page has been even more popular than the home page lately.  Another interesting piece of data is that almost 60% of that article traffic has been from mobile devices.  This is dramatically different from the 16% of home page visitors using mobile devices.

We created 7 experimental versions of the article page to compare with the control version.  Visitors were randomly shown one of the 8 versions of the page and we considered it a success if they clicked a link on the page.  We had a 12% success rate on the best performing version of the page, compared to only 1% with the control version.

The key to success was the inclusion of a list of related content displaying other articles about Shelby Stanga and other local reality tv show stars such as Troy Landry from “Swamp Men.”

This related content module was created manually for the purposes of this test.  Theoretically, it should be possible to create a single related content module and position it on  every article page.  As long as the content is tagged in a meaningful way, the articles displayed in the list should be relevant, thereby increasing engagement and pageviews across the website.  Ideally, this will help increase advertising revenues and magazine subscriptions.

We recommend creating and maintaining a set of meaningful tags for your site content, and positioning dynamic mixed content modules that will use those tags to automatically display related content.

We will be conducting more research to help optimize reader engagement.  For example, we may measure the effect of various positioning and styling considerations to see what combinations are the most effective.  Check back soon to see what we’re up to.