Manuel DaCosta: I am an Experimentation evangelist that stands on rooftops, screaming out. You should be testing better. Oh, you shouldn't be testing more efficiently. How's it going for you? Not very well. I think I need to come down on the ground and talk to people
Bart van der Meer: [00:01:25] this way. You're sitting on the chair right
Manuel DaCosta: [00:01:27] here right now, and then later I'll go out and tell them about that.
That makes sense. Hopefully they'll stick around hopefully.
Guido X Jansen: [00:01:34] And, why?
Manuel DaCosta: [00:01:36] I got invited to the Dutch. CRO awards. Yep. And I've prepared my talk in English clues should have been in the name,
Guido X Jansen: [00:01:45] It should have been,
Manuel DaCosta: [00:01:46] it should have been, I should have brushed up on my Dutch so far. Facebook autocomplete has been helping with it, but I don't think it will help me with my talk.
Guido X Jansen: [00:01:54] Fair enough. Fair enough. So what are you going to talk about?
Manuel DaCosta: [00:01:56] the thing I'm talking about is how. without revealing too much immediately before the talk, isn't it?
Guido X Jansen: [00:02:02] Yeah. That's what the previous guest said too. But I have a pretty strong feeling that people listening to not
Bart van der Meer: [00:02:09] know. And I'm guessing that they're not in the room actually listening to us.
Manuel DaCosta: [00:02:13] That's why the connection crashing. Yeah.
Guido X Jansen: [00:02:16] Everyone's on the wifi, listening to the live stream in the same location.
Manuel DaCosta: [00:02:19] Yeah. so the content of the talk is. it's about conversion optimization to the next level. everywhere, where we go in our own little bubbles, our communities, our Facebook groups, our conferences, et cetera.
There's a whole emphasis on the technical aspects of CRO and the experiments that we've run in the uplifts and stuff like that. But I think there's a world out there. That conversion optimization is still yet to make a big impact on which is the business they work in. And you might say, yeah, but they are doing some really good tests and stuff, but more and more, what I keep hearing is that conversion optimizers are complaining about, we don't have buy-in, we don't have dev resources.
how, why am I getting asked about, to prove my ROI every time? Those are questions that are coming from what I've. Thought of as well established CRO programs. So why is that happening? And so then this talk, I'm going to explore that and also touch upon this topic of culture of experimentation, which I believe is a waste of time.
But if it's done incorrectly, as it is being done right now, that's when it's a waste of time. It's let's say, you are a weight loss guru, a diet guru says, how can I lose weight? eat healthy. Great, but how do I tell you? He tells me nothing, there's no kind of direction and stuff.
So over the last two years at effective experiments, we've been observing and working with our customers and people that we know and trying to really understand what works and what doesn't work, and to create a structured approach to getting this culture of experimentation and addressing. These problematic recurring problems that come up every time buying the resource.
And because all of that is the symptom of the problem, which is well, CRO is conversion optimization is still asylum within marketing, within a department. And my job standing on top of that rooftop is trying to shut and get people to see that there is actually a bigger business that needs to understand the value of it because I'm a firm believer in experimentation.
It is a driver for innovation, but we can't be, just be scratching the surface. We need to be doing more. And so that's the content of the talk. Going to be sharing the framework called the cave model. Oh, nice. Yeah. I couldn't come up with anything else. and, and
Guido X Jansen: [00:04:42] rooftops, caves,
Manuel DaCosta: [00:04:43] rooftops, caves subways next year.
Yeah. so that's basically what I'm talking about.
Guido X Jansen: [00:04:48] Great. and so do you have the feeling that, zero specialists are approaching it wrong? Are they. Not approaching, not looking at the culture or doing it wrong, or
Manuel DaCosta: [00:04:59] they're doing it wrong. no one goes into a job to say, yeah, I'm going to do it wrong today.
They're going in with the best intentions. And as I said, the gurus out there that the people out there that are putting on blogs and stuff are basically talking about the right things. Like we should be growing a culture of experimentation. We should be sharing stuff. We should be, getting people involved in our testing programs.
That is all well-intentioned stuff, but when they go out there and try and do it, they basically in this boardroom where people are yawning and completely unengaged, And this is not just a problem for CRO it's a problem for SEO. It's a problem for PPC. And the key thing that we forget. And as I said earlier, where we're in a bubble, We sat in this bubble where we are patting ourselves on the backs and say, wow, you did a good test. Oh, you did a good test. Look at my test. And everyone's we've elevated ourselves with this false sense of amazement that we were doing stuff. And my friend, Tim Stewart has a really good take on that as he says, We see our role.
We think of ourselves as Avengers, right? The superheroes, but we're actually, we should be agents of shield, which is the low level people that are doing the hard work to get the stuff done. And so we need to take our own egos out of it and really make that difference in the organization. And going back to
Bart van der Meer: [00:06:17] that's my question.
What's the main problem with. Let's say having one full time, CRO, in your company and then doing tough testing things out, doing user research and then sharing that.
Manuel DaCosta: [00:06:28] what's the main issue with it? The main issue is this, right? You mentioned you were from an SEO background as well, is that correct?
Yeah. and let's say you went and let's say three of us, A PPC and SEO and a CRO walks into a bar or a cafe or yeah. Yeah. I see what he did there. See what he did there, but, Like you could tell me all about your tags and your technical. So you're going to be like, okay, now listen to my amazing stuff.
And then the PPC person. Okay, good. Now listen to my amazing stuff. What's happening over there. You're listening in one year, and then you're just trying to shout out about what you're doing. That's basic human behavior. You want to show how valuable you are, right. And the other person putting it plainly doesn't care.
This is human behavior because
Bart van der Meer: [00:07:13] everyone sees thinking what's in it for me.
Manuel DaCosta: [00:07:16] You got one of my slides on there. So that's my story. That's fine. They're not watching it. Yeah, exactly. So what's in it for me. Yeah. So this is why people in that boardroom are there. The other, another slide don't share in this talk as well is sharing stuff, right?
I've done all this work and sharing, putting it on dashboards, putting it on email reports and stuff. If you, if I walk past massive TV screens with stock market information, it's being shared. Yeah. But I don't know. It's been
Bart van der Meer: [00:07:46] being transmitted.
Manuel DaCosta: [00:07:47] Yeah. It's been transmitted, but it's not being transmitted there because I don't follow the stock market.
I don't have any understanding of the stock market and I have no interest in the stock market, so you could put it up all day and I'll be like, whatever. So this is it. It's the same thing that's happening in organizations when we're being told to share. All we're doing is we're trying to get the stuff out, push it out there.
And people are walking past because they've got to run to their next meeting. Yeah. So it's not that they're doing it wrong. It's, they're missing a few steps before that. Think of it. if you're, playing a man to run a marathon, You're planning to run a marathon, do you start running straight away, full speed or whatever you take different steps, your nutrition, or whatever.
Everything needs to happen before you're fit enough to go and run that marathon. And what I feel is it's not that they are doing it wrong is they've missed out. A lot of the steps are I'm going way back. They've missed out a lot of the steps before that, which is
Bart van der Meer: [00:08:39] which results in people not caring.
Manuel DaCosta: [00:08:41] Correct? Correct. So I, as I said, that sharing stuff is great, but it's
Guido X Jansen: [00:08:46] it looks like we have a winner.
Bart van der Meer: [00:08:50] Thank
Manuel DaCosta: [00:08:50] you. Alright. Thank you.
Bart van der Meer: [00:08:53] Thank you.
Manuel DaCosta: [00:08:54] Thank you.
Bart van der Meer: [00:08:56] I don't know. As one.
Manuel DaCosta: [00:09:00] I'm not saying
Bart van der Meer: [00:09:01] it's not a slide. And so I don't.
Manuel DaCosta: [00:09:03] Oh, I think I'm,
Bart van der Meer: [00:09:05] who's a rocket.
Manuel DaCosta: [00:09:06] Yeah. Okay.
Bart van der Meer: [00:09:15] Yeah. so would you, say that, for example, the differences between a CRO running a test about, I don't know, a unique selling point and then not
Manuel DaCosta: [00:09:25] saying
Bart van der Meer: [00:09:26] to different departments like SEO, like we found the issue is being is better or worse in these cases than this one, or is it, Saying we found a dish USP works better than this USP.
And maybe for SEO, you can do this or that with it.
Manuel DaCosta: [00:09:41] That is the second approach is a lot better. But again, it's about getting people involved even before that. Cause again, it's, we ran a podcast conversion nations. I'll just plug that in anyways. in that podcast there was this guy called Alex Abel, and he said really.
That problem, why what's in it for me can be solved really easily while I say easily, it does take time, but you have to go out and actually talk to people, not about your stuff, but about learning about your stuff. So when I'm sat with you, I want to learn, about your SEO work. I want to learn about what your KPIs are, what your challenge, we're making conversation.
We're building that connection, but more importantly, I'm understanding the context with which I need to approach you with. Yeah, because then if I come in and say, Oh, I found this USP. It's because I have learned in that conversation that we had, that is important to you. And if I can help you meet your goal, you will help me meet my goal.
And so conversion optimization. There's two aspects of it. There's the functional aspect. You can't be without that
Guido X Jansen: [00:10:41] A bit. But it's mainly about, doing that with, your colleagues on the same level, or is it also
Manuel DaCosta: [00:10:46] a everyone in the organization? at some point or the other, And that's what takes time and that's what takes the effort.
So there's two aspects of it. There's the functional aspect of experimentation, which is you're running an AB test doing your research, writing up your report and all that kind of stuff. And the second aspect, which I feel is not being done enough or done properly is the transformational aspect. Because what is conversion optimization at the end of the day, at the core of it?
What is it's about moving your business, improving the business. It's innovation. It's about business transformation. It's not about moving widgets on the page. It's not about if we think as optimizes, right? When we're optimizing a page, we optimize it for the benefits. We're not optimizing it for the features we're optimizing for the benefit.
Similarly, the work we do, it's not about, yeah. We run some AB tests. Yeah, we're helping the business, understand the customers. We're helping move, the revenue move up, helping them retain estimates and so on and so forth.
Bart van der Meer: [00:11:41] Even it can go as far as creating new business models or adjust existing subscription model, you have
Manuel DaCosta: [00:11:47] completely correct, but.
The challenge that we face now. And this is something that Jeffrey Eisenberg who coined conventional said back in 2016, when I ran convert or conversion world, he said, conversion, optimizers are seen as like the dirt under the fingernails people. And they haven't been given a seat at the table where C-level makes all those big decisions.
All those big decisions are made. But where the conversion optimizers over there,
Guido X Jansen: [00:12:14] it's weird because there's already a C in the name.
Manuel DaCosta: [00:12:16] yeah. So we started out
Guido X Jansen: [00:12:18] with the three letter acronym, starting with a C we started out,
Manuel DaCosta: [00:12:21] Hey, that's it that's a different topic, right? because we had this whole thing about.
Should we be calling it CRO and then we said, no, we should be calling in bro business rate optimization. Yeah. And it's terrible
Bart van der Meer: [00:12:33] acronyms going around.
but that's for the, the, the bigger companies, the guys that have teams for this and what about
Manuel DaCosta: [00:12:40] the smaller company? And so here's the interesting thing when I've talked to smaller companies and, you'll have one zero and stuff, they feel.
That they shouldn't need to do all this stuff because they're small. They don't really need to do that. Yeah. But my, my honest answer to them is you need to do it even more. You need to do it even more because if you want to scale up, if you have any ambition to scale up, and if you have any ambition to bring people on, you need to be setting those foundations down ASAP.
You can't just say I'm going to do the technical work. It's important. There's no two ways about it. But to make it accessible, there's a lot more that isn't being done. And this is, primarily the core thing of my talk. And also going to be introducing this new concept, a word we've climbed based on the findings we've had, and we're calling it experiment ops.
It's a collection. It's basically it defines all the processes, the people and the work that needs that goes in with one aim. Yeah. And that aim is about helping the business, understand the value of experimentation and changing that culture. Because ultimately, what have we seen? We've seen that if there's the right culture, if there's the right experimentation mindset, you will get ideas from everywhere.
You'll get people willing to give you those resources because they know the value of testing.
Bart van der Meer: [00:13:58] But that doesn't, I agree, but doesn't that mean that maybe you're at the wrong conference?
Manuel DaCosta: [00:14:03] Maybe I
Bart van der Meer: [00:14:04] shouldn't you be at a conference for four CEOs?
Manuel DaCosta: [00:14:07] Maybe I should. I did ask you that, right? Yeah, maybe I am.
But what I'm trying to do is trying to inspire this crowd because this isn't a watcher, And they're already buzzing about the good work and I want to inspire them to take it to that next level, because what I don't want to see. Is people being fired because they've not proven the value of CRO. I don't want to see teams being disbanded.
I don't want to see CRO being seen as like second fiddle to marketing.
Bart van der Meer: [00:14:33] Yeah, we do so many to CRO something like that.
Manuel DaCosta: [00:14:36] Yeah. So I want the people in this room to be valued, if they are trying to change the business, they're trying to explore new business models. These are the people they should be approached.
How can we test this business model? How can we improve it right now? If all we're doing is moving stuff around the page. We lend a, becoming a commodity. SEO went through that. I don't know where it is now, but it did go through that first page of Google, all that kind of stuff. And so as CRO as well, we might end up being the same thing, like a commodity, and that's what I'm keen to avoid.
Guido X Jansen: [00:15:09] So what's the happy end of your story.
Manuel DaCosta: [00:15:12] The happy end is that framework and it's given them some action points. Yup. Seemed really
Guido X Jansen: [00:15:17] happy. The happy end is you should do more work.
Bart van der Meer: [00:15:19] Yeah. ingrained CRO into entire business and you as an CRO consultant or as somebody who works in that field is the first person who should start doing
Manuel DaCosta: [00:15:31] that.
Even as a team of one, it's your responsibility to set the ground rules and said the set of the foundations for it, just because you're small doesn't mean that you and a lot of it. Is not to do with like hard work. It's all about being disciplined and setting those structures, setting the processes.
Because lot of times where I speak to people and effective experiment, we're all about process. We're all about structure and stuff. And when people approach us, they're like, we don't have any structure or it's all over the place. Why is it all over the place? we didn't really, at that time, it didn't worry us.
And. That's my point. It's like that early on. If you're, if you don't think about it, when it's, I compare it to this analogy to noon down that road and trying to change it as you're going, That's not possible.
Bart van der Meer: [00:16:16] Yeah. You have no idea what we can do with
Guido X Jansen: [00:16:18] bicycle.
Manuel DaCosta: [00:16:20] Your Dutchies. Yeah. Cool. Yeah. So that's essentially the message that it is a positive thing.
I'm not trying to talk down any people. There's some good work being going on over here.
Bart van der Meer: [00:16:30] And do you think it would help if a CRO, for example, when they come up with ideas or prioritize or things like that, basically go to the SEO team or PPC or whatever team and ask them, what are your struggles? And can, maybe I can run an experiment to help you get an answer.
Do you think? Oh, definitely.
Manuel DaCosta: [00:16:48] Definitely. you want to do it. You want to do it really casual first rather than go on and tell them, what do you want? So the casual things is again, those coffees or lunch, and learn sessions. Keeping it really casual. And the agenda is like knowledge sharing, right?
And you start with that, you don't start with work, you start with making it fun and then gradually they will come to you and they'll say, Hey, I've got this problem. Can you help me validate this? Then, you're onto a winner and here's the thing it's not going to happen overnight. You'll have lots of coffees.
You'll have lots of lunches and everything, and you may not see any change. But it'll happen over time. That's what I said. It's a discipline and maintaining that consistency. And that's where you'll start to get.
Guido X Jansen: [00:17:28] When you talk about it takes some time can
Manuel DaCosta: [00:17:32] take years, man can take years, but this is what I feel like CRO when they join an organization that has to be a plan.
For longterm with that Tiara, unless there's something really bad about that person and they're really rubbish or their job, that person should be definitely many years. Yeah. Because you can't transform a business with a one year contract.
Guido X Jansen: [00:17:55] but that's something we need to change on the management level.
So that's not something that single
Manuel DaCosta: [00:17:59] person
Guido X Jansen: [00:18:00] that has a six month or 12 month contracts.
Manuel DaCosta: [00:18:02] Nope. You can't do that. And that I feel those type of contracts are something that as a CRA should avoid as well, just because it doesn't give you an enough breathing room to make change on the testing level, let alone the transformational level.
Yeah. So you put yourself in a corner or already, I know it's a job and people want that job. And I might be being a bit naive and idealistic and saying, don't go for those jobs. But the only way we can make that change is by setting those rules. if you think of it, how are people hiring right now?
Because if CRM is so new, right? Management's just Oh yeah, we needed a CRO because they can increase our conversions and stuff. That's the mindset. They don't know any better. So when you said I should be talking to CEOs and CMOs. Yeah. I think I should be because I should be helping them understand how to hire better, how to set those expectations.
It was what to expect from the CRO. How long should they be working? That's the key over here. And so the change has to happen on both levels. It's not like just these guys changing the way they work and their management's whatever.
Bart van der Meer: [00:19:04] And helping companies become, avoid becoming the next Kodak by really understanding what they do, what visitors value,
Manuel DaCosta: [00:19:11] that's the thing.
and, I, hopefully as I said, these guys can be inspired to take it back, but if we start on this level, we can't change that level sometimes, immediately we're trying to do business transformation, It doesn't happen overnight. But if these guys start putting in those. practices in the company, the management starts seeing it.
They start becoming more aware of it. And once you unlock that awareness, that's when you can start making that change. Yeah. Yeah.
Guido X Jansen: [00:19:36] Does that? I can imagine that at effective experiments, it's a tool it's bragged management tool for CRO. and it can imagine that, when, I think that's not our awards show,
Bart van der Meer: [00:19:45] I think
Manuel DaCosta: [00:19:47] probably, yeah, we're gonna go and jump into a song or something
Guido X Jansen: [00:19:52] that would be nice.
So monumental. when companies, need your tool, if they already have a CRO program, and they think, okay, everything is really messed up. We need something to fix this, then they come to you. But I can imagine that's the, door's not the
Manuel DaCosta: [00:20:08] solution. It's not the solution. And I've also told them that because sometimes, and I've turned people away because of this is because they come in with that mentality of Hey, we need this automatic solution that can fix us.
And I tell them, no, our tool is centralizing stuff and it facilitates stuff I want. The people in this room to be valued. And if they are trying to change the business, they're trying to explore new business models. These are the people that should be approached. How can we test this business model?
How can we improve it right now? If all we're doing is moving stuff around the page. We lend to becoming a commodity. SEO went through that. I don't know where it is now, but it did go through that first page of Google, all that kind of stuff. And so I see our role as well. we might being the same thing, like a commodity and that's what I'm keen to avoid.
Guido X Jansen: [00:20:58] so what's the happy end of your story.
Manuel DaCosta: [00:21:00] The happy end is that framework, give them some action points. Yup. Seems really
Guido X Jansen: [00:21:05] happy. you should do more work.
Bart van der Meer: [00:21:07] Yeah, no people just
Manuel DaCosta: [00:21:09] leave after that. I need to get some more work done.
Guido X Jansen: [00:21:12] Maybe you should let them get drinks first.
Manuel DaCosta: [00:21:15] yeah. Yeah. That'll be a good, that'll be a good thing.
Bart van der Meer: [00:21:17] So basically the, the final message is try to. engrain CRO into the entire business and you as an CRO consultant or as somebody who works in that field is the first person who should start doing
Manuel DaCosta: [00:21:30] that. Even as a team, you have one it's your responsibility to set the ground rules and said the set of the foundations for it, just because you're small doesn't mean that you and a lot of it.
Is not to do with like hard work. It's all about being disciplined and setting those structures, setting the processes. Because a lot of times where I speak to people, I think active experiments. We're all about our process. We're all about structure and stuff. And when people approach us, they're like, we don't have any structure or it's all over the place.
Why is it all over the place? we didn't really, at that time, it didn't worry us. And that's my point. It's like that early on. If you're, if you don't think about it, when it's, I compare it to this analogy of if I'm your tire bursts, When you're driving or cycling in Amsterdam, it's easier.
It's easier to change the tire on the side of the road. Then continuing down that road and trying to change it as you're going. That's not possible. Yeah.
Bart van der Meer: [00:22:26] You have no idea what we can do with bicycle
Manuel DaCosta: [00:22:31] Dutchies yeah. Cool. Yeah. So that's essentially the message that it is a positive thing. I'm not trying to talk down any people.
There's some good work being going on over here.
Bart van der Meer: [00:22:41] Do you think it would help if a CRO, for example, when they come up with ideas or prioritize or things like that? basically go to the SEO team or PPC or whatever team and ask them, what are your struggles? And can, maybe I can run an experiment to help you get an answer.
Do you think that's a
Manuel DaCosta: [00:22:57] good thing? Definitely. you want to do it. You want to do it really casual first, rather than go on and tell them like, what do you want? So the casual things is again, those coffees or lunch, and learn sessions. Keeping it really casual and the agenda is like knowledge sharing, right?
Yeah. And you start with that. You don't start with work, you start with making it fun and then gradually they will come to you and they'll say, Hey, I've got this problem. Can you help me validate this? Then, you're onto a winner and here's the thing it's not going to happen overnight. You'll have lots of coffees.
You'll have lots of lunches, everything, and you may not see any change. But it'll happen over time. That's what I say. It's a discipline and maintaining that consistency. And that's where you'll start in the winners.
Guido X Jansen: [00:23:38] When you talk about it takes some
Manuel DaCosta: [00:23:40] time can take years, man, and take years. But this is what I feel like a CRO.
When they join an organization, there has to be a plan. For a term with, without Tiara, unless there's something really bad about that person and they're really rubbish or their job, that person should be you definitely many years. Yeah. Because you can't transform a business with a one year contract.
Right,
Guido X Jansen: [00:24:06] but that's something we need to change on the management level. So that was something that single person that has a six month or 12 month contract. Nope.
Manuel DaCosta: [00:24:13] You can't do that. And that I feel those types of contracts are something that as a CRA should avoid as well, just because it doesn't give you enough breathing room to make change on the testing level, let alone the transformational level.
Yeah. So you put yourself in a corner or already, I know it's a job and people want that job. And I might be being a bit naive and idealistic and saying, don't go for those jobs. But the only way we can make that change is by setting those rules. if you think of it, how are people hiring right now?
Because if CRM is so new, right? Management's just Oh yeah, we needed a CRO because they can increase our conversions and stuff. That's the mindset. They don't know any better. So when you said I should be talking to CEOs and CMOs. Yeah. I think I should be because I should be helping them understand how to hire better, how to set those expectations, what to expect from the CRO, how long should they be working?
That's the key over here? And so the change it has to happen on both levels. It's not like just these guys changing the way they work and their management's whatever.
Bart van der Meer: [00:25:15] And helping companies become an, avoid becoming the next Kodak by really understanding what they do, what visitors value.
Manuel DaCosta: [00:25:22] That's the thing.
and, I, hopefully as I said, these guys can be inspired to take it back, but if we start on this level, we can't change that level sometimes, immediately we're trying to do business transformation, It doesn't happen overnight. But if these guys start putting in those. Practices in the company, the management starts seeing it.
They started becoming more aware of it. And once you unlock that awareness, that's when you can start making that change. Yeah. Yeah.
Guido X Jansen: [00:25:47] Those that I can imagine that at effective experiments, it's a tool it's brag management tool for CRO. and it can imagine that, when, I think that's not our watch,
Bart van der Meer: [00:25:58] I think the stream is
Guido X Jansen: [00:26:00] probably
Manuel DaCosta: [00:26:01] yeah.
Where he's going to go and jump into songs.
Guido X Jansen: [00:26:04] Yeah. It would be nice. but Manuel, when companies, need your tool, if they already have a CRO program, and they think, okay, everything is really messed up. We need something to fix this, then they come to you. But I can imagine that's the door.
It's not the
Manuel DaCosta: [00:26:20] solution. The tool is not the solution. And I've also told them that because sometimes, and I've turned people away because of this is because they come in with that mentality of Hey, we need this. Automatic solution that can fix us. And I tell them, no, our tool is centralizing stuff and it facilitates stuff.
We're not promising a silver bullet that you put into your system. And then all of a sudden it's like magic. We're not that type of business that promotes that. Yeah. We help them because it centralizes stuff. And we see that by centralizing, it takes away the time. So they have more free time to do evangelizing and so on and so forth.
But really they need to put in the work. Yeah. So I can spot, depending on, the first few sentences they say, and what they're looking for, I can tell whether they are ready for it or not as well, regardless of what stage the company is at.
