Over the past month, our team has been running tests on landing page conversions, this very newsletter, and native advertising.

For fun, here are the tests we're running up top. Take your best guess at which versions won and I'll spill all the deets below.

This month's tests: 

LANDING PAGE CONVERSIONS

• Standard sign up form vs. Fast lane chatbot

NEWSLETTER FORMAT 

• One-column format with section headers vs. Four box grid with content and images for each link 

In each case, who do you think won? The answers may surprise you. Before you read below, take a minute to make a guess. And then scroll down a bit further for the results. 

Landing Page Conversions

What we did: For a community sign up page, we tested two versions of the same form. One was a standard sign-up form on a landing page, and the other was a chatbot that would ask you if you wanted to join the community. 

VARIATION 1: Traditional form on a landing page

This first version was pretty straightforward with a traditional Marketo form at the bottom of the page. Visitors would have to scroll a teensy bit to get there, but nothing too crazy.

VARIATION 2: No form, chatbot sign-up

The second version skipped the form altogether and instead replaced it with a chatbot in the bottom right-hand corner.

Why we did it: We ran this test to see if the form-filling experience could be impeding sign-ups and ultimately whether people would prefer submitting a form in the way they're used to or enjoy a more human-like experience like you get with a chatbot. So we removed the form entirely on the chatbot version and instead gave people two ways to sign up—via a simple "Want in?" message in the bottom right corner. Or in a sign-up button in the banner. Our team thought we'd be able to replicate some of the success we've seen with other fast lane chatbots that allow visitors to skip the form, but...

Results: That was not the case. In fact, the traditional form produced a 36% conversion rate over the chatbot which saw a mere 13%. However, in the chatbot version that "Sign up" button in the banner produced a higher number of clicks (probably because those visitors weren't crystal clear on how to sign up via chatbot). When they did so, it opened the chatbot fully (see below) so it became clear what needed to be done.

Learnings: Ultimately, the traditional form on a landing page won out here. We didn't see a significant boost by using the chatbot in this test because we pitted them against each other. Perhaps if we had included both versions (which is what we do now) our total number of conversions would ultimately be optimized.

Newsletter Layout 

Okay, we're getting meta here. And if you've been with me since we launched this newsletter in May then you may have a good idea of what I'm about to say next. Don't @ me for sharing a test that's not from this (or even last) month. The results are still valid, I promise!

What we did: We tested two different layouts of our Be Better Newsletter to see if one format would outperform the other in terms of click-through rate. For its first two months, we ran the same content in two different layouts.

VARIATION 1: Straight-forward one-column layout

The first variation focused on the copy in a straight up and down one-column layout. There were no limitations on the number of content pieces to include or sections to add. The design was kept minimal, clean, and simple.

VARIATION 2: Four-box grid layout

The second variation included less content in the actual newsletter as the four-box grid layout only allowed for, you guessed it, four content links.

Why we did it: When I revamped how we communicated with our subscribers and doubled-down on authenticity and strove to create a monthly email that felt personal like I was a marketer talking to a friend or coworker (Fact: I have this relationship with one of my coworkers and aimed to share the same types of inspiration, challenges, test results, quotes, with all of you that we did with each other each month). My manager disagreed with this approach and wanted something much more stylized and heavy HTML. We met in the middle with the four-box grid.

Results: Over the two months we A/B tested the newsletter format, the four-box grid layout netted out with less than a full percentage better click-to-open rate. It's not a huge difference but ultimately I made the call to switch to this layout because the four-box grid layout appeared shorter on mobile and desktop and the results didn't warrant the creation of two newsletters each month.

Learnings: Sometimes your gut is wrong. And the only thing irrefutable is data. Be open to trying, testing, and experimenting because it's a sure-fire way to produce results.

I hope this inspires you and your team to run a few tests.