Guido X Jansen: [00:00:00] Today, we're going to talk about service with them and told them about online dialogue. My name is Guido Jansen. And welcome to the book where I showed you the behind the scenes of optimization themes and thought with their specialist about data and human driven optimization, and implementing a culture of experimentation and the better in case you missed it.
In the previous English episode, we spoke about copywriting for conversion, the jolt cleft key. You can listen to that episode zero K website pouring your podcast. As you're listening with polite now. This episode is made possible by our partners online influence Institute, content square covert.
SiteSpect an online dialogue. Welcome to season two 36. Great to have you both back in the cafe and let's start off with our definitions. What's the difference between service size and hindsight's testing.
Ruben de Boer: [00:01:07] if the client's like testing is fine site, the source, like testing and service.
Guido X Jansen: [00:01:11] Okay. So let's explain to the client and the surfer for those who don't
Ruben de Boer: [00:01:14] to make it really basic with blindsides.
you almost always get a tool like optimize to PWB tasty, where if a client side solution replaces scripts, on your websites, mostly in the hat. And when you experiment way run AB tests, the script of your test is being loaded on the browser, on your website. Pfister where as wanting to serve her sides, you need to play some scripts on your server.
And your test runs on your server. also codes you bill for your variation is noted on the server, which means you're one up, see a flickering effect for instance.
Guido X Jansen: [00:01:48] And, Tom, what will be the, distributional service that's first client site does that you guys currently perform at? Or mind dialogue
Tom van den Berg: [00:01:55] question?
I think 80, 20. So 80% client sites, then you would send server-side.
Guido X Jansen: [00:02:03] Okay. So what are the biggest differences? when making the choice. Okay. We have this experiment that we want to run. We have our, both CS and now we come to the stage. Okay. We need to decide, if this is even possible with client side testing, which is often already available.
Or do we need to go further with this and do something that's more on the service side do and maybe even implement the tool for that. so how would that decision process go? What are the dependencies? Yeah, it's
Ruben de Boer: [00:02:31] of course we need it badly Fest. You want to run, but basically all it's tarred with, checking the maturity of the organization within perverse rates optimization experimentation.
if you're just starting off with lawn CRO specialists and a lot of development capacity, if any. then investigators do client side and you have more easy tests. Whereas if the, if there's more mature organization, more mature, if it comes up question for redesigned tests, backend test pricing tests, the answer is definitely easier and better way to go.
Tom van den Berg: [00:02:59] And it's also, you are in a better position in the organization, I would say. So when you start alone is zero, especially the STR most of the time. There's running desks at your own. And besides is, there are still a development team was doing a lot of development stuff and that's never tested actually.
And when you have the opportunity to do circumcise testing, you can actually test all your releases as well. And that's, I think a good thing because then you can really see what the effect is for all the stuff you are building. Yeah, but that's mostly for more mature organizations, I would say.
Guido X Jansen: [00:03:33] Why would you say that?
So why is it more for mature open, organizations to do surface hides, experiments? is it more expensive or does it require more expertise or
Tom van den Berg: [00:03:42] not sure if it requires more expertise, but I think that the organization needs to be ready to test everything. And I think that's something different, right?
Because like normal organization, it just build to off, they think they know how, Customers will react to staff. They think they know that everything they build we're actually as value through the physics of the website, but that's actually not the case. And I think when a company is ready to test everyday and to, yeah, to do actually, yeah, be ready to do this everyday.
And I think, the client side is more of a starting point or fell CRO and AB testing. And so our side testing is most of the times becomes later in the process of the majority.
Ruben de Boer: [00:04:25] Yes. Love to see it that, if you start off, let's say you're just starting off with the client science, so you can have your first AB test up and running in one hour.
And you just need to base the script on your websites, which is like five minutes for a developer. And you set up your first test. If you do server sites with more people needed, maybe there's also illegal and I should be in full. So there's been more work to get it started. And so therefore the answer is yes, on for, for, experimentation should be a bit higher before we start cervicitis
Guido X Jansen: [00:04:53] sounds a bit like, you're saying that with service side, you can test everything while with client side there are limitations.
Is it also, is it truly the case or are there also scenarios where you can, are there things that you cannot test service or that you can do client side?
Ruben de Boer: [00:05:08] Good, not a service I think into clients. And now I would say no, it's mainly with server side, you can run more complex beds, like test algorithms, orders and checkouts.
especially the checkout change, the order of that. big features, product testing, that's possible the server side testing clients like testing. There are some hacks, but it's fairly difficult. And if not impossible,
Guido X Jansen: [00:05:30] can you give us some examples or maybe experiments that you guys did that were not possible client side, but that you, needed to implement service?
Tom van den Berg: [00:05:37] Yeah, actually prior to this skipping, the shopping cards glides, right? So when someone's on the products page, they click on a bicycle and we just wanted to move in, bearing into the checkouts because the average products for all animals really low, I live with more than one. So we thought they just need one product.
Why do you actually need to show up in cars? So we want to do the client sides, but what happened on the background was that the people who are actually visiting the shopping cart, but you didn't saw it. So it was like a big load I'm on the back on the back of the loads. I've never, still. I think the shopping cart and also that they went through the checkout because it's not possible for insights or actually remove that stuff because it's built in because, so we tested this and we saw that again with when you saw the experience of the visitors in the variant, it was actually really bad.
So we go there. This was actually useless. Yeah. so now we're trying, we're going to try it again, but then serve so you, yeah, for really false load with the IB, actually skip the staff and then you can actually say if it's really influence behavior. So I think that's a. Good example one Oh one I have is that, for example, a new navigation with S with one's a clients, so navigation on the whole over and all the websites.
And did you want to do with blindside the low diagram gumbo? a longer. So we test that server sides and then actually you saw no difference in load side, then you're going to really make sure that the difference in behavior,
Guido X Jansen: [00:07:02] you're saying that showing in addition to being able to test different things, or at least more things with, with the service side, there are also additional benefits, like low time.
are there any other additional benefits with destined serve side
Ruben de Boer: [00:07:17] another example, which matches your question is braces redesign. you have a, companies, websites that's pushing a redesign live straightaway without testing it. it makes sense because I've never seen it happen. I've never seen the work blind sites.
I've seen a lot of tries and efforts, but it never worked blindside. Whereas, if we a server sites, you can test the whole website, you could even test different backends, to validate your redesign. so I think that's another big advantage and something you should do when you do redesign, you should test it server side.
Yeah.
Tom van den Berg: [00:07:48] And I know that she . So I talked to a bank and they didn't want to implement a client site. Maybe they're still in the minor hating, let's say. Because the developers are getting a bit nervous and security departments that actually you use that actually do great the service sites too, because he just said, otherwise, I'm going to implement the client side.
And they said, no, we're going to do it because all the client's information again. Yeah. Go to the tool.
Guido X Jansen: [00:08:19] Yeah, because basically what you're doing with clients site testing, you're placing the average script cookie that is usually out somewhere else. So basically someone else, while you can, technically, you can host it yourself, but most of the times that's not ours, how it's done, how it's implemented.
So you embed a Jabra scripts from somewhere else from VWO or Optimizely or whatever tool you're using, which is then loaded in your, in the browser of your customer. And yeah, you don't have any control over there JavaScript. technically they could change it. And if they have a security issue in their code somehow, and someone can change that maybe, even a deck in the middle and change that to alpha script.
technically they can do stuff on your website. That's you might not want, or as a bank.
Tom van den Berg: [00:09:01] Yeah. Or they get access to the data.
Ruben de Boer: [00:09:03] Exactly. And then again, there are different rules again, for wherever the server is the testing tool. Where did in Asia or Europe or United States? so clients are definitely legal.
also has a say
Guido X Jansen: [00:09:14] marketing bridges have suffered. And the sheriff for AB testing has been impacted too. If you want to keep that thing to enterprise standards, but safe, 80% on your annual contract, you can consider comfort, but there some release you can take advantage of full stack and hybrid features, strong privacy compliance, no blink.
And enterprise great security feel good about your smart business position, infest what you saved. Becking your CRO program. Check out the www adults over slash 2020 while the, I think since I'm over a year. Browsers are more and more vigilant than in removing cookies or being more strict in using cookies.
Is that a additional benefit for serve sides or does they'll make a difference?
Ruben de Boer: [00:10:03] No, definitely. With service side, you place a so called first party could piece. whereas with client sites, you place a third party cookies, like you just mentioned, and, browsers are a lot more acceptance towards first party cookies because you have the data for herself and you don't have it with some third party.
So definitely that's a Bethany big advance as well for service side
Tom van den Berg: [00:10:26] in the future to give me more, I think for different browsers, maybe it does now. I think Safari's doing it and yeah, you will not know what happens in the next year because privacy's yeah, God was more and more important. I think.
And I was saying that is, I think also that, if you build that service side and it has a positive result, it's, it should be really easily implemented because you're already built that there's it's a part of the GMs or the actually what actual websites. So it would be one personal or both then to actually implement it.
That's not always the case, but, we see some problems with companies that they that's a lot, but there's problems with implementing winners. And this should be raised here with whichever side. And that's a good thing
Guido X Jansen: [00:11:09] because it already exists.
Tom van den Berg: [00:11:10] Yeah. So the clients side is actually.
it's actually far off the website.
Guido X Jansen: [00:11:19] Would you say that also means that it usually takes longer to build surface side experiments, because you actually need to build it and make it fully functional. Whereas on the client side, you may be able to cheat some things, be able to validate your hypothesis, but not actually build the functionality.
The way it should be built. but, maybe take some shortcuts there. So
Ruben de Boer: [00:11:42] eventually that festival no easy, desperate clients' sites by ISTE set up. And if you have a limited knowledge, some base models on front end development, you can do it yourself, but it's a desk that's bigger and on Sarah's side would be a, the more easy way to go.
I don't wanna say you have to build such impossible hacks, which not always work. so the best little size of a desk
Guido X Jansen: [00:12:02] and, in your experience. So would you say that Glines side that's also less effective usually on average, compared to serve shutters because you can do more elaborate tests on side?
Tom van den Berg: [00:12:14] Yeah, I actually would say so. I also think that inclined site testing, you also have the figure of figuring out which also going to be at test itself, Loaded the experiments a little bit later, then default
Guido X Jansen: [00:12:26] for those that don't know, I hope everyone knows, but there are, and maybe you haven't noticed that the, with your tool so good on you if that's never happened, but.
often tools that lost clients are low. If the connection is a bit slow or the browser is a bit slow, or I don't know, you have a very slow computer with a, and your CPO CPU is overloaded. It might happen that you just load the original page. And the other script kicks in a couple of seconds later. Or maybe if it's really horrible, Then seconds later or even worse.
And then the, the variance appears, or changes the page or whatever is your, a variant. And that's obviously not a good thing, and that's not something that happens on service.
Ruben de Boer: [00:13:07] Exactly.
Tom van den Berg: [00:13:08] And I think, at a type of days you can do like price testing or releasing products and actually that's what the impact is.
I think that's yeah, more connected to the business companies are doing then some of the easy line site testing, Sales on competition are doing. Of course you can go mind boats, or you can maybe have one thing for example, is doing still decline site testing, which is easier for example, to also improve your website and small steps, but also use service testing for the bigger stuff.
That's also an option.
Ruben de Boer: [00:13:36] Yeah. And I think if you have server side testing, you can test the bigger stuff. And I think more and more people will be start to get interested, within the company for experimentation. Because she starts touching the products or the real, the value proposition or algorithms.
and I think those big, impactful tests, only you to perform those, you get more and more people at Suggestic within your company to start experimenting or at least see what's going on and see the possibilities of experimentation. so yeah, secondary benefits of service testing would be to get more people on boards, more colleagues on board.
Guido X Jansen: [00:14:10] So are there a sort of the tools that you guys have experienced with, or that you see emerging in the market? Would it be more, tools that are integrated in the CMS that people are already have, or maybe even in the analytics tool that they, have service sides is hard, those AB testing tools, experimentation, those embedded in that, or are they're all.
separate tools that you, install in addition to what you already are.
Tom van den Berg: [00:14:33] I think before that are two, types of companies. You have one that actually wants to use service testing also to encourage people to test more, but they actually build the O'Toole whether UI on it and they want to make sure that every product team is really started test.
So you get an overview of how many tests are running and what the win rate is and everything. And they, the, I spoke to a couple of other smaller companies and they actually only built the technology. So Dave has server side solution with no UI. It's only like technical stuff, but they just wanted to build it because it's the desk Ardon way better.
And they confessed everyday. And it actually started with a question like, I want to test my search algorithm a versus B. but I can't do with blindsides. So vinyls builds the solution ourself and it actually was way easier than I think the only loss. They actually use that solution to run all the tests now.
And there are some other companies who are, yeah, just using it to make experimentation bigger in the company. And they actually build a whole too around it. And they also integrated the analytics part. But if you start it easy, you can, yeah. You can do the analytics still in the own analytics, in the Google analytics, for example, or another week.
And just connect those with the server side solution. You have
Guido X Jansen: [00:15:46] the AB testing part, whether you do as client side or server side, it's not that complicated, right? You just need to be able to, divert your traffic. in a decent split between the variants that you've set and do that, in a decent random manner.
but so server-side tools are all they exist for a long time now also, but I think w we can, it's a fair assumption or fair statement that the client side tools were very popular last 10 years and are a bit more mature than the service side tools. so you guys been experimenting with, server site tools more and more.
Is that a fair assessment? are there already things that didn't serve size that you cannot do or not yet do or, like usability issues that you see there? Are they indeed a lesson?
Ruben de Boer: [00:16:33] No defense there's organizations can either choose to build for themselves with your IRA without UI. I use interface, but schools like AB testing, optimizing.
I also have their solution for server sites with their grades. Do I, with the experience they have from their client's site tool, but still with service sites, you are as if you're a CRO specialist, you're probably more dependent upon development. then you are, when you've been, we built test flying site,
Guido X Jansen: [00:16:57] what kind of skills would you need in addition to running a test on clients?
Because if you run desk, client side, you already need to know a bit of HTML, CSS, JavaScript. so you're saying that's not sufficient to do that, to run an experiment service,
Ruben de Boer: [00:17:12] Server side, actually a code into the original code of the website. So the fancy and what language your website is written.
I need to know deco, which could be a bit more difficult than the basic HTML, CSS and JavaScript or jQuery if you prefer.
Tom van den Berg: [00:17:26] Yeah. And it can also be harder to find someone to actually be part of the team, because from that diversity, you just need to find one. Then they actually can all build the best.
But if you have a specific boat used on your methods or decode language, which is hard to use in Holland, for example, and it's really hard to find a developer for building test. And then maybe you asked too, but they also building a lot of other stuff. Then it's hard to find someone where actually it's going to help you build an AB test
Guido X Jansen: [00:17:54] fondant developers are way more common and there, they all know HTML, CSS and JavaScript.
Almost a while back end can be that this is a very diverse landscape and the kind of languages that you need. So you're pretty, you're probably needs a for service at Dustin. You probably need the developers that actually build the website to also make your test show you're fishing in the same pool of resources.
As say, as the rest of the company, while for client sides, you might, you might be able to do it yourself. Or it's fairly easy and relatively cheap to just, get a, get an outsider and to build the test for you.
Ruben de Boer: [00:18:29] Yeah, exactly. And if you are a zero specialist and build plane sides, I think you can learn front end development in couple of hours, maybe two days.
Yeah. It's not that hard to run a basic tests. if you decide to do the small and medium complex tests yourself, and you can learn it in two days and get the complex test of development, whereas if you go to the service side, you always will probably need to developments if you are a zero specialist, but I see also see with clients that's that dependency, the betterment is increasing ones.
websites go to SBA single page application. So for instance, reacts angler, it's harder to test on those blind sites tools. Don't always work. so that's another reason why you should, yeah. Good move to server side testing. and again, more and more dependent on the betterment in that case.
Anyway,
Guido X Jansen: [00:19:16] SiteSpect offers a worldwide unique AB testing, personalization and product recommendation solution. SiteSpect works the service sites without any tags or scripts, which guarantees and ultimate performance, the size Ellucian eliminates delays and the chance of bickering effects. And this approach also ensures that the current and future browsers, skirt your rules, like you feed them eat.
Don't make an impact on your AB testing and personalization. Have more info visit dot com. You've helped companies, while doing dope and doing those experiments, but also implementing, the service sites, tool sets. What are some common mistakes or roadblocks that you see with companies, that are implementing surface experiments?
Tom van den Berg: [00:19:59] I think that many companies think it's really difficult to actually build the solution. I spoke to one company that actually discussed, that they want to set this up. Like we should go sheriffs, like those thing, are you sure? And they made all like the pros and cons and discuss a lot about it. And then they started it.
And also three weeks there at the basic version in life. And off the six weeks, we just have one developer, our once a year, a specialist, they actually have like a. Yeah, pretty decent service side solution ready to actually be used. yeah, I think ServerSide's also, actually at the vendors. They, a lot of people don't know a lot about the, the service side solution, like the marketing guys at the vendors, they all use blindside and server side.
Also. It'd be, I think it's as a gyno. I dunno, name or feeding that is really difficult, but what you thought would be better for us? It's actually not that hard. So yeah, our advisors would also be to just start using it. I started actually building it and see iceberg.
Guido X Jansen: [00:20:54] So w what you always, so those tools do I always.
so the more common tools that offer our service sites, testing, do they always work with different kinds of, programming languages or does it all depend on the programming language that your size is built on?
Ruben de Boer: [00:21:10] I haven't run into those problems yet. I wouldn't exactly know if it runs in all languages.
I would assume it does own the most popular
Guido X Jansen: [00:21:16] languages. If it doesn't then of course, then self built will be probably the best solution for you.
Ruben de Boer: [00:21:22] And start let's start, like Tom said, start easy. Just, start with the technology itself. Maybe not yet the user interface and from bearing you can, they can keep on building and then started user interface, get more data service side.
you can start off easy with just the technology
Guido X Jansen: [00:21:38] when talking about the technologies and vendors. of course there is that while we already mentioned, AB tasty, isn't a solution, a PWL full stack Optimizely full stack. Of course we have SiteSpect as a solution. which is a bit different.
I think, SiteSpect is a proxy server or the intercept methods. so they put a server between, your website and the customer, for serving out those different, Varian's I think has a, solution for this, But w when I was researching Dell solutions, I came across difference, different definitions of server, side testing.
I think some of the, referred more to a hybrid model. technically, if you are an, a client sites, AB testing tool, all of them also offer a split you're out testing, and you could argue that is also a form of service sites testing, the actual variants. you just use a, the other script to point your users to different solutions.
So that might be some kind of a hybrid method. so what are your favorite technologies or vendors for doing this?
Ruben de Boer: [00:22:43] The question that I, first of all like that, like the splits, your role, like resolving, to each other before you started recording, there are some hacks you can do at client sites, which.
It's then kind of service, like for instance, for refreshing brands, I wanted to test algorithms, for recommenders. So we basically had two recommenders on the product page one on display, none one on this way, block blindside, they beat us with a simple CSS change. We switched them around. So we showed the other recommender and the way we test the algorithms, that isn't a kind of nice heck.
but for tools I have, I don't really have a preference yet, but I've seen some good
Guido X Jansen: [00:23:20] solutions. Yeah. I also did. I think it was five years ago, 40 algorithm. There was this, when I came on board, they already volunteers, algorithm running, to optimize the category pages and the tool allowed for, basically testing one algorithm versus another.
So you could, you could test them those, but they didn't have an sort of, was already built in and it didn't really have an option to turn it off and actually. That's the algorithms against doing nothing. And I think I judged, I just used a, the client sites, AB testing tool just to basically rip out the algorithm tool, to do nothing, to disable it and then run that against each other.
Tom van den Berg: [00:24:00] Yeah. The funny thing lady, we did some, I did some research on it was driven on different companies and we actually found out that most companies are building it themselves
Guido X Jansen: [00:24:08] as in your clients, the,
Tom van den Berg: [00:24:11] and some of our clients, but also these companies we're actually using or considering service side and mostly.
Most companies actually develop this solution internally instead of using a, one of the defendants we just discussed
Guido X Jansen: [00:24:26] on the one hand they will, that would indicate, okay. It's actually quite simple to do it yourself. So maybe that's so good at it. On the other hand, I would say that it still sounds a bit inefficient, right?
It should. why aren't there server side solutions. that I can just pull off the shelf and implement for my company. is it a technology issue? Is it, are those all really expensive to do? Or what do you think is causing this?
Tom van den Berg: [00:24:47] Yeah, I think the actually just building the solution can be quite easy, but then of course you still have some maintenance also, to actually keep on using the technology. If you build both yourselves. So maybe just start up, can be really cheap, but then you also have maybe one developer. actually you're maintaining your own solution. And if you ever blind, or if you use a tool for that, maybe you've made it for the tool, but you don't get the maintenance.
So I think it's kind of balance between those two. I actually, yeah. Find out what's the best solution. And maybe also the cheapest one. That's the thing somewhere in the middle,
Guido X Jansen: [00:25:24] depends a bit on that, on how you're organized development sites within your company, if you can basically do one or the other.
Yeah.
Ruben de Boer: [00:25:31] And again, that also depends on the goal. if your goal is to do service side with elements that then their own solution will, probably be doable to build just the technical part. Did you want to do a if wants, allow it for a lot of people in the company to use product teams. then, using a vendor which has the technology already a good user interface based on their experience with clients sites and data implemented, or they have to full solution, but it costs more money.
where as you can start off with yourselves as technology, like we mentioned, and then build from there, build, using face, build the data. So it's also a choice of
Guido X Jansen: [00:26:09] the goals. Yeah. By the way, the, those questions from, the most common mistakes and the favorites, the technologies, those came from one of our other podcasts guests.
He posted those questions in our Facebook group. And if you want it to be, be more involved with, other Ciro specialist, you can go to chiro.cafe/facebook and ask you a question. those are upcoming, upcoming guests.
Tom van den Berg: [00:26:29] Can I add one thing to the, to a discussion we just had? I think. A couple of companies actually told us that it's cheaper to build able solution themselves then instead of using a tool.
So I think that's a, especially to companies, we just built a technical solution, so they don't build a whole year around it. They just use it from a test server side. And I think also if you look at the defenders, for example, sites back, they have a lot of knowledge about service like this thing. And they're actually, they do it for a long time already, although it's not the same exact same solution as the others.
So I think they are a little bit better. And it's also into actually used in their marketing because I think a lot of people actually at the fenders, they don't know a lot of our services testing. It's mainly focused on clients. So I think they can need to step up their game as well. Although they are pretty big some companies and I think optimized, for example, it's really focused.
I think it's good. They can still make something movements. They are actually show how to do with how easy it is. I give more information actually. So you use cases. And that's maybe also yeah. Majority, which need to grow on that kind of stuff.
Guido X Jansen: [00:27:38] What'd you say that, in terms of data quality for your experiments, or is there a difference between client side and server testing, which is it's so is there any benefits for growing server-side?
Ruben de Boer: [00:27:47] So if you, do everything right server sites, then you get
Tom van den Berg: [00:27:51] promoted.
Ruben de Boer: [00:27:52] Of course, that's always there. Important that they've said about everything, And there, I can give you more trustworthy data as its role data coming from, from the server instead of if you send it to Google analytics and make sense of it.
so I hear about cases where a suicide data is more trustworthy than clients,
Tom van den Berg: [00:28:08] that data
Guido X Jansen: [00:28:09] for over 10 years. Now, all my diet of advisers about evidence based conversion optimization with a focus on data and psychology. We see that analyzing data and recognizing customer behavior results in a better online dialogue with your clients.
In a higher ROI that team of strategists, analysts, psychologists, and UX specialists gathered valuable insights in your online behavior of your visitors and together with you optimize the different. So if you're a hero program through redesigned expert reviews, AB test and behavioral analysis, for more information about their services, go to Owen dialogue.
Yes, sir. You're not missing out on anyone. That's blocking alpha scripts. Yeah,
Tom van den Berg: [00:28:49] exactly. But then still you have all the raw data and then you can need to make it analysis around this. So the bad thing, or the negative part can be that you actually rebuilding your analytics to Nexstar yo who had X, which already have, and then you have your own server side solution, which are analytics tool that you're actually selling kind of products or your website, It's not like you are a kind of vendor for products. I think you need to be. careful with it as well. Otherwise you spend a lot of development time and people on actually. Yeah. Building stuff, which is already out there. Yeah. So it's a, yeah, I think the data going to be better, but maybe even with Garfield, with it rebuilding your analytics solution,
Guido X Jansen: [00:29:27] open stores, tools out there on GitHub that people can just pull off the shelf and then try to implement it for the wrong company.
Ruben de Boer: [00:29:34] Facebook has the open source. I've had a service sites. I'm not sure about any others right now.
Tom van den Berg: [00:29:40] I'm not sure if it's. Open source, but split on iOS as a free solution. It's not open source, but usually you can also use for sharper side.
Guido X Jansen: [00:29:48] I'll do a quick Google. Yeah, I come across Facebook and there's this wasabi.
Have you got,
Tom van den Berg: [00:29:57] yeah, I've heard about that. Yeah.
Guido X Jansen: [00:29:59] any, things that you guys are working on right now to make sure for science experimentation better to make it better, but you guys run a lot of experiments. do you need to rebuild? I know you've guys built, automation for, for experiments, saw the whole jar.
event lately with, Tom sharing things, how he, automated, the reports from experiments, with air table. Is that something that you now completely need to rebuild? when you do a service side testing, or is it something you can also reuse
Tom van den Berg: [00:30:29] and we can still use that. especially if he doesn't make the analysis in the tool or your builds herself or from fender, but if you still use your ethics, so usually for debts, that actually doesn't change.
I think once you do most of that, mostly now, is that the discussion with our clients, if it's actually, they should, we should, they need our analysis. because I think lo time becomes more and more important and nowadays. I think Google was also, yeah, focusing more on that. So you're actually results and Google are influenced by a lot of them with your website.
And if you put a lot of scripts on it, it actually becomes slower. And the experience we thought was better for the Vista we use server-side so there are becoming more and more volunteers or actually doing it. So that's why we also start to discuss it with our clients. And it depends of course level yeah.
Stage of maturity. Also the maybe divergent diaries and, the status of CRO or AB testing inside the company. So it's been more and more discussed. Actually let it last months, I would say called Barracuda four or five years, I've worked on that before. So that's a kind of a development for that way.
Ruben de Boer: [00:31:32] So the best way helping, clients, choose the tool, and find the term together. If the client side is sufficient for now, or they switch to server sites and how they should do
Guido X Jansen: [00:31:43] so for new clients or clients that are reevaluating their bedroom, are you more and more recommending service either, or maybe even only recommending service side by an hour?
Ruben de Boer: [00:31:53] I would say so. But again, it depends on the maturity appliance.
Tom van den Berg: [00:31:56] Yeah. Yeah. Or it's a culmination. So some Fest we're doing service sites, which are a bigger, more important for the company, for example. And we're still testing line size next week. Oh, it's actually a funny story. I heard lady, when y'all have Shiro D made her flight scientist and there's a developer building test and it uses you, your own mindset issues too, as like between 20 and 30% of the experiments is positive.
But when you start having server site testing, you're actually gets thoughts of different developers. And they are used that Darko is always going live. But now it's actually tested. So then they see it as a 30% of this working. So they actually throwing 70% of their code away and that's not really good for their mindset and, I'm motivated.
They get, so we are from different companies that they actually are that actively working on, keeping their developers, still building stuff. Although actually 70% is thrown away, maybe. So that's a different mindset and that's actually changed maybe culture or behavior, which is pretty interesting.
Guido X Jansen: [00:33:00] I can imagine that your approach can be to develop new functionality that those developers are building, and doing that more in smaller steps and keep validating if it is actually adding value to a user instead of. I'm building a school, new functionality are working on that for six months only to have it shut down by a, by an experiment, two weeks after.
that's obviously not, not nice for those, those developers, but if you can build in small steps and then. Early on. You can quickly already see, Hey, this is not working or we should go a different path. The chance of something not working, over a six month period is a much more, for such a large project.
I think,
Ruben de Boer: [00:33:41] yeah, I am. I have the bills, MTP, the smoke tests and step by step.
Guido X Jansen: [00:33:46] Exactly. So how do you mentioned this earlier that you can now with turf start testing, you can actually experiment on a full release. Do you already have clients that have this implemented? So how would that process look like?
So there's a release ready? Maybe there's a twice a month or maybe even a weekly release off of their website with new functionality. do you do a full split test between the or the previous version and the new version? Or how would you approach it? Yeah,
Tom van den Berg: [00:34:12] so we have, one client I'm working with we're actually doing in service sites.
And the cool thing is that one of the people from zero time is also part of the development team. So she's like the owner and both teams. There's some desks are part of the sprint they actually doing and ever release every three weeks. And it just going live with release on, for example, Monday.
So then when, it's boost life, they actually push an AB that has life instead of the whole, or is that the go through to our numbers down to visit and goes to 50%. And actually the AB test is starting. And we already implement the end dates when it goes live. So it goes out automatically and we are force monitoring the results day by day.
So it's actually how it works out.
Guido X Jansen: [00:34:55] How long do those experiments run
Tom van den Berg: [00:34:57] red diamond? The minimal detectable effect we have. So we actually galvanize France. So it's more planning because yeah, you can.
Guido X Jansen: [00:35:05] So a minimum detectable effect, if it's just a release that's I don't know. it's not always based on all my proper hypothesis, whatever they are they're implementing.
They're just like fixers or I don't know, additional functionality that has to be done because legal or. Or whatever. So how would you determine that
Ruben de Boer: [00:35:22] indeed? you prefer I'm the MD, that's not always the case. I just finished a project with, with a big multinational company where we, desperate the whole website, but front end and backend, service sites, the backend was really outdated.
So logistics, wasn't as good documentation finance order follow up front and also needs some feature fixes. So they had a complete. New backend front ends on different server. I, we did a server side and that worked well. we analyze the data tweak and make sure that the conversion rates don't drop, when you have pushed the whole release new release, new website life.
yeah, that's definitely an example. Marcel recite is just perfect, to work with and, make sure you, have low risks when you do a
Guido X Jansen: [00:36:04] full release, but you, each week it can be something different. Of course, that changed that candidates, especially as service side, you can change a lot more things than you experiment on client side.
So I can imagine that's the only, like the main KPIs that you just basically check on live. They're not changing too much, but you cannot check every variable and that's out there.
Tom van den Berg: [00:36:25] That's where we're at. And I think it's not the case that most companies we're working with the everything. So there's a release.
And maybe when it's something really small on a page with a lot of festers, then yeah. You can still test that. You can also think, okay, the impact will be really low. So just put it live. It's not like booking does go home that they test average after you back or everything, especially on breaks. She's right.
There's a lot of effects in the order flow. That's the, yeah. The thing we're focusing on. And then we actually Fest it server side, although it's a new subscription model or a new pricing stuff, or a total different redesign. From that page. These are battles because otherwise, in Africa, everything would go on to the best also on the FAQ page, blah, blah, blah, or whatever, like 10 people a day.
Yeah. Maybe you should just put it live and. Yeah. It's two years and then you will be there as well. But
Guido X Jansen: [00:37:16] yeah. Yeah. I'm just saying it's more like a check that things don't go horribly wrong when you release something. And that's when you want to
Ruben de Boer: [00:37:24] now, if you have a server side technology, you can do feature flagging feature rollouts as well.
so it doesn't have to be in big experiments, but if you can roll it out, 5%, 10%, 20%, and see if there are no bugs errors, it's nothing.
Guido X Jansen: [00:37:38] Yeah. Especially if it's, if those are big changes, you want to maybe gradually release them by 33 audience.
Tom van den Berg: [00:37:45] I think it's big in the rest of made as part of support of AB testing.
Is that what you're saying is not losing money instead of, you're only looking for winners, but in this case you can also, I want to talk to Lucas from Brooklyn about it. He also said that's the, actually the main founders for booking is that they. For sure. They're not losing money with everything.
What's that already big difference. And then of course you want to add some extra revenue also by finding a positive result. I think many companies that don't know what they actually do that area
Guido X Jansen: [00:38:15] yesterday's brainstorm was so good. I really liked step's idea of running that test on the call to action buttons, making them orange will really make them stand out.
Don't you? I think. Yeah. Do you want to design real AB test winners and achieve enormous conversion uplift, then stop, brainstorming and take a scientific approach. If you can read Dutch, follow the steps and online input the bestseller on the menu, even book Delta now and rule in the office course and become an expert in applying proven behavioral signs yourself, go to dot com for more information and free downloads.
Yeah. If you're, especially if you're AB testing capacity is limited, then you can only do so much experiments in a month. While on the other end, the developers are releasing a new stuff. Like every week there's an, there's a new release. It's we call it in Dutch development economy, mopping the floor while the is still running.
but, yeah, so you're. you're you can be very limited, but you want to make sure that those releases don't break any big things or make things worse for your customer. but there's no intention to do that, of course, but, you might to make sure that's not the case.
So I just, so one last, I think is something that's a benefit for service site, that things, since you have all the data in house, I think. Almost by definition. you have, since you have all the data from individual customers and, it's easier, a non binomial test, surface rights. you can easily more easily compare revenue gains for four experiments when you run them server side, first client side.
If you do client side, usually. you're working with aggregated data and Google analytics, and then it's not possible anymore to, to compare those
Ruben de Boer: [00:40:04] true. when I was working for a publishing a website, we need to have the Nominom, your KPI
and we have separate scripts within each test to send the client ID to Google analytics and do it that way. but yeah, so resides, those should be more easy
Guido X Jansen: [00:40:20] to get that data. Darryl has to do with, client side, but, it needs additional work. So I want to make a small side of that. like Don just mentioned, of course has speed is very important for your indexation.
so that might be a reason to run server sites. it does seem to, cause it's a bit faster, recently, Google announced that, they also want to include like UX factors in their ranking. Have you guys read anything about this?
Tom van den Berg: [00:40:43] I actually did it
Guido X Jansen: [00:40:45] so they want, they want to include those in the ranking, but I'm very much wondering how they, how on earth they will.
it needs to be automated. They're not they're notes. Having a user panels, evaluating every site in the world. So I think, so there must be some way to automatically, judge the UX of a website. I'm very curious how they will be doing that's
Ruben de Boer: [00:41:04] fine. And in some sense, it's already in there of course, because if you search for, if you have a search term, like wanting to buy a computer and you click on the first results and within a minute, you're back at Google and click on the second results, then Google already knows that the first result lessons that relevant for you.
And of course when you as well, Beth contributes to people finding the information they need and buying a computer they want. So in some sense, I guess it's already in there.
Guido X Jansen: [00:41:29] There's definitely an overlap that, so for this, for the 40 effectiveness of the patient is definitely the case. If you're back in Google, then that's an indication, Hey, apparently that person didn't find what they were looking for.
So they're back. but, so since they announced this as a separate thing, that they will be looking at the user experience, I'm very much I'm intrigued about how they will be doing this.
Tom van den Berg: [00:41:52] I think they just need to crawl every website on how many AB desk companies are exited rotating. So they know that's going to be actually checking once it's working better.
And then they shoot. Here for a higher score for death website,
Guido X Jansen: [00:42:05] then I just go about and just run a million experiments on my websites, not caring about the results
Tom van den Berg: [00:42:12] I haven't gone really slow. So you still
Guido X Jansen: [00:42:13] end up doing it service side. So that's all fine.
Tom van den Berg: [00:42:17] yeah. Yeah. Yeah, it feels this way. So let's you're I don't want to fix that.
Guido X Jansen: [00:42:26] I th I think you can measure it. I just think it's going to be really hard just by and by running a script on a page. So any other things you guys are looking forward to do, to change or improve in the coming 12 months, whether it's serves site testing or any other general things that you're working on?
Ruben de Boer: [00:42:41] Of course. Yeah.
Tom van den Berg: [00:42:44] Yeah. Yeah. But I'm still working on also, as we were in evades Elma, I'm like getting more and more of the process and our Dave will actually, so in the latest thought he thought uric, we actually included them. And now this is off the test. We also not implementing the research. So to the phase before the experimentation, so that it becomes more and more, Yeah, kind of platform or all the information or the knowledge of optimization is there, but we all need to build a AB splits on top of it.
And then we have our own tool. Maybe there'll be cool with the next year
Guido X Jansen: [00:43:15] and the air. So the air table talk. I will link to that in the show notes. that's I think there's like a five minute talk. It's very, it's quite short, right?
Tom van den Berg: [00:43:22] Good way to set it up yet. It's five minutes dogs, $5 a day for five minutes.
Exactly. Yeah.
Guido X Jansen: [00:43:26] For five days, four minutes and 51 seconds. You're very good
Tom van den Berg: [00:43:32] because she already recorded herself. That diamond was next.
Guido X Jansen: [00:43:41] talk faster and faster towards the end door. Basically in short, you, you show the solution where you use air table, where people can log their experiments, and then you're even using the API, to get Google analytic data in a fire. Google sheets. Yeah.
Tom van den Berg: [00:43:58] Yeah, that's correct. So we put the coolest sheets.
So the good thing is that everything of the process are these I, and I want to say is all the information needed to actually get a test from through Z that is already in our table, but you also want to create a database with actually a results. And we all combined us and everything and will not. And the next step is to actually also with the research we do out from the ed.
So you're actually doing experiments on the Bates, for example, but you don't do that. You also have their friends in size from the own page and it would be cool to get one overview of the experiments, your surveys, the data, heat maps, whatever, everything together. In one place
Guido X Jansen: [00:44:41] and automation of the reports, of course.
very helpful. Very nice to save a lot of time on there.
Tom van den Berg: [00:44:45] Yeah. I think it's, it stays around, two to three hours per experiment. So yeah, that helps you
Guido X Jansen: [00:44:51] if I'm, if you run a couple of experiments a month, for sure.
Ruben de Boer: [00:44:55] Yeah. You have all the information in one place, which is a, it's a big benefit.
Exactly.
Guido X Jansen: [00:44:59] For a nice. Guys, thank you so much for, for sharing your experience with, with, surface site testing and we'll link to the tools mentioned, we'll link to the, I had some other blogs on this topic. If you want to read more on this and as a blog, all my converse at comb CXL and, They all have written some stuff about this lately, so I'll make sure to check those out.
And of course I'll link to, to talk, from the whole jar that lightening talks about three tips to improve your experimentation program with Ansible. Thanks. Thanks. Thanks much guys. Talk to you soon. And this concludes season two, episode 36 of the sheer Oak fade podcast with Ruben, the board and dialogue next week in our English episodes, where we'll be discussing how you can and should be here using experimentation to manage risk and create a risk profile for your own or your client's business.
I will be doing that together with the CEO of conversional Steven, talk to you then, and always be optimizing.
