This interview was recorded several months ago, but is still relevant despite the pandemic. Colin Nederkoorn, the co-founder of customer.io has taken a unique approach to building their company. Customer.io does marketing automation for the entire customer lifecycle. They have raised funding, but not traditional venture money, and they’ve run it more like a self-funded SaaS. Colin and his cofounder John left their jobs with no savings, and they set out to build an analytics tool. Their story is powerful because of their unconventional approach and ability to persevere through hard times.
The finer points of the episode:
- 4:05 – The customer.io founder journey
- 5:23 – Their approach to selecting investors
- 7:01 – Reflecting on how Colin and John bootstrapped a SaaS app after leaving their jobs with no savings
- 8:02 – Why they pivoted from an analytics company to selling marketing solutions
- 13:15 – Finding the balance between innovation vs following the best practices
- 18:37 – How customer.io became a remote company, and the advantages/disadvantages of building a remote team
- 22:05 – What customer.io is doing to support the bootstrapping startup community (and why they care about bootstrappers)
- 24:30 – Marketing approaches that customer.io used in the earlier days
- 31:55 – The highs and lows of building customer.io
Items mentioned in this episode:
Rob: Welcome to this week’s episode of Startups for the Rest of Us. As always, I am your host, Rob Walling. Thanks for joining me this week. Thanks to everyone who reached out about last episode 495 where I went solo. I got more positive feedback about that episode than I have in six, eight, nine months. I appreciate folks reaching out. Let me know what I’m doing right and let me know what constructively could be doing a bit better.
In this episode, I air an interview that I did months ago. It was certainly pre-pandemic and it may even have been before the end of last year. While there are no mentions of COVID or Coronavirus in the interview, I think there are so many lessons learned from the journey of this founder, Colin, the co-founder of Customer.io. Customer.io has taken such a unique approach to thinking about how to build their business, the way that they got it started, and the way that they didn’t go down the venture track but also didn’t straight bootstrap. They were one of the first companies that I had ever heard doing that.
Before we dive into that conversation, if you haven’t heard of helpfounders.com, you can head there. It’s a collaboration between a bunch of podcasts that are intended to help Bootstrap founders and folks who may be impacted by COVID or maybe it’s just an effort to give back, so different podcasts. A lot of us in the bootstrapping space offered up just a couple ad slots but really it’s just more of here’s this company. Here’s what they do just to make the Startups for the Rest of Us listenership aware. This is all voluntary. It’s a non-paid sponsorship. It’s really to give back to the community.
The company I want to talk about this week is called Hugo. It’s at hugo.team. According to the founder Darren Chait, he says Hugo is centralized, searchable meeting notes that connect with tools such as Zoom, Slack, Zendesk, and HubSpot. It’s free for up to 40 users. The target market is SaaS companies of all sizes including brands you already know such as Atlassian, Shopify, and Spotify. They were a good addition to the other work-from-home tools that are growing in popularity. If that sounds interesting, head over to hugo.team.
With that, I hope you enjoy my conversation with the co-founder of Customer.io, Colin Nederkoorn.
Today with me on the show, I have Mr. Colin Nederkoorn from Customer.io. Colin, thank you so much for joining me.
Colin: Hey, Rob, great to be here.
Rob: I’m actually pretty surprised that we haven’t had you on the show before now because you’ve just been in this bootstrapper/ these days, I’m calling it an indie-funded space where folks are raising small rounds but they want to keep control of their company. They want to stay independent. I really feel like you were one of the first, if not the first companies, to do that. I think we have some really good stuff to dive into today.
Colin: Thanks. It’s interesting because I’ve never felt like I belonged in any community, but certainly I share more values with the bootstrapper community than I do with the ‘venture-funded grow at all costs’ community. I guess there’s that great quote from Groucho Marx. I forget what the quote is.
Rob: He says, “I would never be a member of a club that would have me.” That makes sense. As I’ve watched you, certainly, you want to build a real product for real customers who pay you real money, which is something I often say on the show. That is much more in line with this non-venture track startup software.
For folks who haven’t heard of Customer.io, you guys effectively built out and launched over the course of 2012. You really had your public launch in 2013. According to your website right now, your HTML title tag is marketing automation for the whole customer lifecycle. Your headline is, “With Customer.io, you can send targeted emails, push notifications, and SMS to low return, create stronger relationships, and drive subscriptions.” I know you started off doing a lot of emails but now you’re in push in SMS.
In 2012, you guys designed your first logo. As we know what’s important in SaaS is not building a product and selling it to people. It’s designing a logo and printing business cards in 2012. I want to ask you a question about that in a second. I want to get to the timeline so folks can keep it in their head.
April Fool’s Day of 2012, you guys had your first full-time days working on this. You quit salary jobs with no runway, five customers paying you $10 a month, no savings and no income, which we’ll get into in a second. This is great. You can’t make this stuff up. You wind up raising about $225,000 from friends and family later that year. As I said, you launched January 2013 with between $5000 and $10,000 MRR. That’s when you raised a seed round.
You extended that friends and family around to $750,000. You’ve never taken truly venture money, institutional money. It’s been more from people who are willing to just support your vision of building a profitable company not go Unicorn, go IPO, or go home type of thing.
Colin: I think that’s pretty true. I don’t know if I could say that we articulated that well in the early days. I don’t know that we had those values but I don’t know that we self-selected investors based on that alignment. I imagine some of our investors, the other investments that they made either have died or are really big companies now.
Rob: Has that caused you any issues in terms of investors who did want you to go Unicorn? Some investors don’t want a profitable business or they don’t want whatever at $10, $30, $50 million exits. They only want Unicorns.
Colin: Not really. We have about 40 people on the cap table which our lawyer always tells us that’s a lot of people on the cap table. Anytime we need to get people to sign documents, it’s a big headache to get everyone to sign. What that’s meant is that for the people who are investing in a lot of high growth startups, we’re not in the front of their minds. Early on, we got a lot of help from people, but now we don’t hear from them too much. They have a big pile of money. They make a lot of investments. We’re just one of them.
Rob: Today, you have more than 1400 paying customers, 57 remote employees. You mentioned you have a small office in Portland with about five people but effectively a remote-first company. You told me offline that you have been public with revenue recently, that you guys are doing $11.5 million ARR, and that you grew 32% last year.
I wanted everyone to have the context as we go into this conversation to hear about your journey, truly bootstrapping this in 2012, to then raising a small amount of funding to get you to profitability, to help with growth, but never taking the massive plunge that a lot of folks do. I’m curious. You and your co-founder John left your jobs without savings, without income, expecting to be able to make this work. How did that come about? These days that would be an anti-pattern because we know SaaS takes forever. What was your thinking back then?
Colin: I can’t even understand why we did that. It just doesn’t make any sense reflecting back on it. Our milestone that we wanted to hit to go full time was 5 companies paying us $10 a month. How you get from there to full-time salaries is a pretty big leap. I think we just knew that we had to do it and then we’d figure it out. We’d do some consulting or we’d start doing services for companies and be able to charge more. But if we didn’t take the leap, then we wouldn’t be in the ocean. We’d still be on the beach and we’d still be trying to figure out how to take the leap. By throwing ourselves way into deep water, it forced us to figure out how to survive and what we needed to do to realize the business.
Rob: You set out to build an analytics platform. What caused you to alter that course and instead go after email?
Colin: We decided in January that we wanted to build the company. I think in the very early days, we spent some time talking with prospective customers. This was right around the time of Lean Startup, Eric Reis’s transformative book. We spent a lot of time going out and talking to people who we thought would be good customers. What we learned was this is what people felt. I still don’t know how true this is but people felt like they had tons of different analytics tools that help them understand their business. The struggles that they were having were they could see what was going wrong but they couldn’t influence it.
We heard that a few different times. We thought to ourselves with that analytics data, we can make something happen. We can send an email. If we’re embedded into a website or mobile app, we actually know what happens after someone clicks on the email and they go back to the app. We can build this amazing circle or we can close the loop (essentially) and show people the impact of the emails that they’re sending in influencing someone’s behavior in your product. That seemed really compelling. When we started talking to people about that, they wanted that because it actually moved their business, whereas giving them another analytics dashboard didn’t.
Rob: Customer is one of the first tools, if not the first that I ever heard of, where the data that you use to send emails could come from inside a SaaS app, a mobile app I believe you focused on. At that time the ESPs that I was using that were popular were Mailchimp and AWeber. I’d vaguely heard of Infusionsoft. There was Constant Contact but those were just marketing places. They were purely about email blasts which are totally not personal, not behavioral, not anything. Customer was the first one where I thought it’s a very clever use. I love how you guys came about that by saying we’re going to use analytics to power email rather than send emails in order to get some type of click-throughs or whatever.
Colin: I think that we were coming from a venture-funded startup. Interestingly, after we heard this from the people doing customer development, we looked back at our company and said, “Oh, wait, that’s the problem here too. Our marketing person wants to send emails in Mailchimp but she asks the engineering team, “Can you please export a list of people who have done this but not that.” Then, there were all of these transactional messages that were in the code of the application that that same marketing person would want to tweak the language in and we would have to put a ticket. I think we were doing Kanban boards at this point. We would write up a notecard. As the head of product and the head of engineering, my co-founder and I would always deprioritize that no card because it wasn’t that interesting to us.
Customer.io, when we thought about it, if you had the data, you could send that newsletter without having to ask engineers to export anything. If you pull the content out of the code of the application, you could make changes to your transactional emails and all of your trigger-based emails without having to ask an engineer to do anything. It was all about strengthening the relationship between your engineering team and your marketing folks, product, or whoever was responsible for talking to customers.
Rob: That makes a lot of sense but it was a big leap. There are obviously a lot of tools that do it today but we’re talking effectively eight years ago that you were working on this. I have one question for you about designing the logo and getting business cards. Did you ever use those business cards because I chuckle these days. I think I printed some on the move for one conference that I went to or something. Was that something you look back on and you were like, why did we do that so early?
Colin: I think when you start a company, it just doesn’t feel real. I think people print business cards and they make a logo because somehow that makes it more real to them. When you tell someone I’ve started a company and you hand them a business card and it has a logo, maybe that convinces them that you’re real, too.
I think it’s just a lack of confidence in a company actually coming out of the other end of this process. People want to start with business cards and so we did as well. We were going to meetups and other things like that like going to social events where you’re like, “Hey, we started a company.” “Oh, cool. What does it do?” “It does A, B, and C. Here’s my business card. Contact me.” It’s so silly but I think it’s like a playing company. It’s part of the package of the playing company. I don’t have business cards now.
Rob: I was going to ask. Do you still have business cards? No. None of us do. Unless you’re literally in person with people at conventions and you don’t want to do the exchange of your conferences, events, or whatever. You don’t want to do that type this into your phone type of thing. There’s just not a ton of reason to do it these days.
I read through several interviews that you’ve done over the years. I like to do that to prepare for these interviews. There’s something you said and one of them struck me. It’s something I wanted to dig into. You said, “So much of what people consider conventional has never felt right to me and our company.” Could you expand on that and maybe talk about one or two conventional things that you guys don’t do or maybe some unconventional things that you do at Customer.io?
Colin: I think there’s really a balance here. There are things that you want to innovate on and there are things that you should try to find best practices for. The things that felt wrong to me was certainly venture scaling a company didn’t feel right to me. Typically, what I would see is most venture-funded startups fail for one. The approach that I would see founders take when they were scaling a company with venture money was that it was really undisciplined and they were just spending cash all over the place. But at the end of the day, it didn’t matter because they would end up crashing the company and landing a job, working as an investor, as an entrepreneur in residence, or whatever, something like that.
It just bugs me that essentially irresponsible behavior can end up with a positive outcome for people but that’s how that world works. I didn’t feel like I wanted to participate in that. Trying to straddle the line between bootstrapping a company and raising money is something that I never felt like we wanted to do either of those things, the conventional way that people do those things.
We’ve explored things like how to organize the company and how to run the company. Holacracy is something that people have experimented with. That somewhere where maybe at this time we want to explore things like that, but I’ve realized that that’s not a place you want to be innovative and challenge the norm. Unless your company is all about how companies work, you should probably not challenge the norm there and just use the tried and true methods that have worked over the years.
Certainly on stuff like running a remote company, we faced a lot of skepticism from early customers. Our now CMO once told me that he worked at a publicly-traded company back then and he had to go to bat for us internally to say that we’re actually a legitimate business because we made this choice to build a remote company and we didn’t have physical office places.
There’s a bunch of decisions like that. I think I’m a contrarian by nature. I typically feel like an outsider and I like being an outsider. I like the challenge that that creates for people. For me personally, I think the struggle is important. Feeling like an outsider means that it’s never the easy path when you look at technology companies.
I imagine a lot of bootstrappers feel like outsiders in technology but the happy path is you go to Stanford because you come from legacy or something like that. You have a legacy into Stanford. You graduate from Stanford. Maybe you work in consulting or you join a startup. Maybe you go to an investor and work at an investor for a while or you work in Google or Facebook. Out of your experience at Google and Facebook, you can go and raise a really big round of funding. None of those things felt right to me or particularly accessible to me. I resist all of it because I see how it works on the happy path if that makes sense.
Rob: That makes sense. I think that’s where the bootstrapper is in that indie-funded path. I think why it has so much traction is that most of us are outsiders. Most of us don’t go to Stanford. Most of us don’t know a venture capitalist who lives down the street from us. Most of us don’t grow up in the Bay Area. I think there’s a real appeal to having a path where you don’t need to know someone to break into it, where it feels like this, where you are an outsider that you can’t break through those doors. What you and John did is you didn’t go the bootstrap route and you didn’t go to that funded route. You pick the third path that so few had gone down at that point.
Now, there are more companies. There’s EDBC now that is effectively funding companies to become profitable. There’s a Tiny Seed Accelerator I run that is funding companies with the option to become profitable. Even though I’ve done about a dozen angel investments on my own, half of those are in essentially fund-strapped companies. Part of the Startups for the Rest of Us’ drinking game is when I say fund strapping. I always say it’s a term coined by Colin from Customer.io. It’s always the same thing. People have given me crap about it because they’re like why do you say that every time? It was this really novel concept back then. I can imagine that trying to stay away from dogma is almost what it sounds like. You didn’t want to go down the dogma of the VC nor the dogma of bootstrap necessarily. We’re thinking, is there another path.
I think remote is another big thing that in 2012–2013 (as you’re saying), people wouldn’t take you seriously if you didn’t have an office. Was that a decision from the start like this is just what we’re going to do? Have you ever regretted that? I’ve had remote companies. I’ve had a non-remote. I’ve had half remote companies. I know the value of being in an office with someone. Talk to me through that.
Colin: It’s hard. There are many things which are harder when you’re a remote company. We didn’t set out to be remote but we knew the value of deep work at that point. When it was just two of us, we were sitting next to each other, communicating with each other in a campfire chat, because I knew that if I had an idea and I wanted to share that idea with John but he was deep in work, I could just type it in the chat. He would ignore it, continue doing what he was doing, and he’d get back to me later. We definitely knew the value of deep work. We set up to support asynchronous and remote really early on.
We chose to do remote out of necessity because I couldn’t see a path forward building the company in New York City, trying to find the people that we needed and be able to attract them at the salaries we needed to pay to compete against other venture-funded startups in New York City, the investment banks, and all of the people who could pay way more than we could. We had to find a thing that we could offer that none of those people could offer.
To me the value of working for Customer.io as a developer or anyone on the team is you get to do the type of work that you would do if you were in San Francisco or New York, except you can be anywhere. Let’s say you get a co-located job in Cincinnati, Ohio, chances are the engineering problems you’re going to be working on there are not that interesting but we have really interesting engineering problems. That appeals to the flexibility of the work. The interest of the work is what was exciting to me and what was our competitive advantage, essentially, to hire quality people wherever they were in the world.
Rob: That’s the promise and the beauty of hiring remote. You don’t have to pay Berry or New York salaries. You can hire in whatever, the Midwest, the South, in the middle of nowhere, in Washington. You’re able to pay to give someone a higher standard of living without them having to have a long commute.
Colin: We couldn’t afford New York and San Francisco salaries out of necessity at that point, but it was never the goal to spend less on salaries. One of the things that we do now, we increase salaries as soon as we could, 2014–2015, we pay market rates now. We benchmark to the US national average. I think it’s the 75th percentile of the US national average for all of our roles. One of the things we’re still figuring out is how to make adjustments against that for international. My philosophy on this is that if you live in a less expensive place, I want to share in that benefit with you, the benefit of a cheaper cost of living. If you live in a more expensive place, we’ll share in the cost as well, but we won’t fully adjust the salary for that more expensive place.
Rob: I think I want to ask one question. There’s so much to talk about because you’ve been doing this for eight years. There are a lot of aspects of the customer that I think are interesting to listeners. You had mentioned to me offline before we started that Customer.io is not necessarily ideal for bootstrappers as customers because of the pricing. It’s $150 a month to get started. You guys want to fix that. You would prefer to help bootstrappers out. Do you want to talk a little bit about what you want to offer folks who are listening today?
Colin: I personally think it’s a little unfair that there are all these offers and opportunities out there for venture-funded startups. AWS has credits you can get and all of the companies out there do that. I want to find a way for us to better give back to that community and better support the bootstrapper community.
I think as we talked about earlier, we started at $10 a month. Over time, we raised our prices. One of the things that we found by raising our prices is a lower churn because if you have a really low price point, many people will sign up but also many people will churn. We were pulled by our customers to make the product more and more sophisticated. Because of that, if you’re one person working on a company, the investment required to use Customer.io effectively is probably beyond what you want to do.
The sophistication in Customer.io is probably beyond what you want to do, but I think that not every bootstrapped company is really looking to be a solopreneur company in the long run. I think for those companies that want to grow, expand their team, and are looking for a high growth rate, we want Customer.io as a product to be accessible to those people so that they get the value that the companies that pay us thousands or $2000 a month. All of that sophistication, they get that available to them on day one of their company.
I think you shouldn’t have to use crappy tools, basically. We think Customer.io is a really amazing, flexible tool. We want more people to have access to it but it takes investment. If you’re a solopreneur, you probably don’t want to do the investment required to get the value out of Customer.io. We’re exploring this right now. If this sounds interesting to you after checking out Customer.io, if you go to customer.io/bootstrapper that will give you a little more information. We want to have a conversation with you and figure out how to give you an offer that really helps you get started in the product.
Rob: Circling back, I want to look at some marketing approaches that you guys used in the early days and how you got traction. Obviously, this whole journey is hard but I think you and I both know that the first three months, six months, nine months are really trying because you don’t know if you’ve built anything people want. People are giving you all different kinds of feedback. People are telling you you’re too expensive, you don’t have the features, whatever, and you’d have this fragile idea. You’ve been working on this for eight years, $11.5 million ARR now, much less fragile. In the early days, you’ve talked about Twitter actually being an early sales channel and that your word-of-mouth was very strong. Can you talk about how that came about, why that worked, and what worked at the time?
Colin: There were a couple of things that happened in the early days for us. One, this area that we were focused on became a pretty hot space for conversation in tech in general. I think this guy, Paul Stamatiou, wrote a blog post about User Retention as a Service. People were just talking about this in general.
What I would do before we had a product to sell, I would see people talking about the problems we were trying to solve on Twitter, I would reach out to them, and I would say, hey, I want to understand this better. Can we talk? I don’t have anything to sell you but we’re working on this problem. That was a way to gather information to figure out if we were building the right thing.
What I found is under the guise of not selling something, trick people into having a conversation with you but really you’re trying to sell them something. That’s happened to me a couple of times as CEO. It’s really annoying but we were genuinely not able to sell someone anything. That made for really useful conversations with future prospective customers.
I think at that time Rand Fishkin was talking about this problem. He introduced me to people on his team at Moz who were working on it. We did a customer development call. They never became a customer but I learned really useful information from that. It’s pretty typical. We had a signup page where people could register their interest. We would send people to that page and we’re getting a decent number of signups there.
The other thing that happened at this time is I think Patrick McKenzie introduced me to this guy Ramit Sethi. Ramit runs I Will Teach You To Be Rich. He agreed to meet us for coffee. He asked what we were doing to build an audience and communicate with people as we were building the audience. I said we’re collecting all these email addresses. In six months when we launch, we’re going to email them to let them know.
I don’t know if these are his exact words but it was something along the lines of you guys are idiots. You got to talk to the people whose email addresses that you’re collecting because in six months when you email them, they’re not going to remember you. Figure out how you can provide value for them in the meantime. They signed up for something. Give them what you can now that will help build your audience, then they’ll at least remember you when you launch the thing that you want to sell them. That was hugely valuable advice.
One of the things that we were able to do—I don’t know how easy or possible it is to do today, because there’s just so much content marketing out there—we built our newsletter list, we started writing about email copywriting like trigger messages, and had a pretty good following of early-stage companies and CEOs of startups basically, one- to five-person startups primarily. That really helped us in the early days. Over time, I wasn’t able to keep myself motivated to continue to write content and do content marketing. That fell off a little bit, but fortunately, we’ve built a bunch of evergreen content and still have a lot of inbound today. That’s really how the word-of-mouth engine got kicked off.
Rob: That makes sense. That really has become a playbook. It was emerging during that time. I’ll say 2011–2013, 2014 content marketing playbook. It sounds like you and your team by this point executed really well on the product, but you also got maybe a little lucky in terms of hitting this thing at an inflection point with targeted messaging event-based behavioral stuff. I think that’s something that I’ve been talking about. We’re really thinking about this concept of what do I think the keys to success are. I think there are just three things. I think it’s this simple. I think it’s skill, hard work, and luck. It’s a combination of those three. One plus one plus one equals success.
At varying times, if you have a lot more luck, you may need less skill and experience if you get lucky to hit it. If you don’t have a lot of luck and you want to just purely do something you can repeat over and over without having to make a billion-dollar bet, then you need skill/experience and a lot of adding. You need a lot of hard work. It sounds like you had a little of all of them because I’m just going to assume in knowing what I know about you that you guys work your asses off. The hard work was there and you had development skills for sure. It sounds as I just said, you get a little lucky with that thing, but it sounds like you’ve executed very well on that vision is an addition.
Colin: The way I think about luck is that it’s not evenly distributed. You can’t just go anywhere into any market. There’s some luck that you can tap into. There are some areas and there are some products that you could build today where you’d be extremely lucky relative to other products.
I think we were really fortunate in the space that we picked. We had a lot of interest at that time. We had a lot of competitors at that time but it was really nascent. We were able to build a very immature product and get some traction. Now, there’s a much larger moat where the expectations of what our product needs to do on day one are much higher.
When we had five customers paying us $10 a month, the reason we were limiting how many customers we could service and the way that all that stuff worked was my co-founder would write a MapReduce script behind the scenes when someone would set up a campaign. They would write in plain English what they wanted the campaign to do. We would manually look at the data that they were sending in to figure out if it was possible. If it was possible, we’d write a manual like MapReduce query to find the right people to match the campaign.
You could not launch a product today with that approach. Nobody would take you seriously. I think our experience working in tech helped us know that that was an interesting space to go after. Our luck in picking that space has helped us a lot to become successful and really just the luck of who we met along the way.
Rob: There’s always a little bit of that in all these stories especially when it’s folks like us coming from the outside who didn’t have some in this space. I’m curious. What’s the high point of building a customer? What’s a moment you can think over here like this is amazing, I love what I’m doing here, and I’m euphoric at this moment?
Colin: I don’t have a moment where everything was so amazing like I’m sitting there with a glass of champagne, I’m looking at some company dashboard, takes over, and then all of a sudden, I drink the champagne and feel just like pure joy. I don’t have that. Basically, every time one of the people on our team has a child, every time someone buys a house, every time one of our customers is really happy with us and gives us the feedback that our support was amazing or they were able to accomplish something in their job that they hadn’t been able to accomplish before, it’s those micro-moments that make me feel a great degree of satisfaction just deep inside. It’s not revenue-driven. It’s what having this company and what having this product allows us to do and allows the people touched by it to do in their lives that create the lasting feeling of success, if you can even call it that, but really just satisfaction.
Rob: How about on the flip side? What’s been the hardest part?
Colin: There have been moments, knock on wood can’t really happen to us anymore, but we made some decisions early on technology choices. I think in 2015, we had an outage that lasted 24 hours. Basically what had happened was we were using this database technology called FoundationDB. It was acquired by Apple. Apple would not let FoundationDB renew any service contracts and they took the product off the market. We were in production with this thing with no migration path. The service went down. We were hosting the database ourselves but our database went down. We couldn’t recover it and we couldn’t get any help.
I thought that was it. I thought we were going to have to call our customers and say, sorry but we can’t get your data back. We can’t bring the service up. You’re gonna have to take emergency action to migrate away from Customer.io. That’s the lowest point I’ve personally experienced in the company. If you can imagine as a business how you could fail your customers in the most dramatic way, that is it. If service goes down all of a sudden and there’s no recovery, no migration path, no time, that’s it.
Rob: That is devastating. My palms are literally sweaty thinking about it because we never had outages that long but I know exactly. I’ve been in your shoes. How did you get out of this? Don’t leave us hanging. How did you figure it out?
Colin: My co-founder was trying a bunch of things. I think we had someone else on the engineering team or maybe a couple more people on the engineering team who were helping him with this. We’re just trying things, trying to figure out how to recover the database. One of the key aspects of this is this was a distributed data store. There were lots of machines essentially running one underlying database.
The big problem with that is one, it wasn’t necessary for the type of business that we had. Customer data doesn’t relate to any other customer data. There was no need to have a big distributed data store to run our business. Basically there was one single point of failure again. The idea of these things is that a server can go down and the database will stay up but the problem is you still have this single point of failure. When a server goes down, there’s data that needs to get moved around and it overwhelms the network.
We receive a ton of data all the time from our customers sending us data. That made it even harder to recover. It got bad enough that on Twitter I was like, “Hey, we really need some help. If anyone knows anyone who worked at FoundationDB, please can you connect us?” I reached out on LinkedIn to a bunch of people who had worked at that company and were now working in Apple. Some people responded to us. I think we got some help from one or two people there.
We were ultimately able to recover this whole time. As soon as the acquisition happened, we were immediately trying to figure out how to migrate away but it was too much data. We couldn’t do it fast enough, but we ended up getting things back up and running and had to really massage that database every single day to keep it running while we created a migration path.
Rob: Wow. These are the kinds of stories that don’t make it to the front page of TechCrunch. It’s the growth curve of Slack and Zoom that you’ve mentioned offline, how Zoom goes public, Uber and all this, but how many people are talking about the realities of what it feels like to grow up.
Colin, we’re coming up on time. If folks want to hear more about you, they can go to @alphacolin on Twitter. Thanks so much for joining me.
Colin: Thanks, Rob.
Rob: Thanks again to Colin for coming on the show. As we wrap up, I have a couple of emails that came in that I just wanted to mention here before I take us out. I got an email from Adam. He was responding to episode 479 where we talked quite a bit about marketplaces. I think there may have been a question about who’s talking about building marketplaces. He says that they have a podcast that covers building their marketplace called Menyu. The podcast covers their bootstrapping journey. If that sounds interesting to you, check that out.
The other email I received was from Justin. He said, “I just wanted to say thank you. Your show has been invaluable to me and my co-founders over the years. We’ve built a tool. We’re revamping and launching in a few months, but everything from figuring out what LTV for our customers will be to evaluate whether a free model could work for us. You shed insight on a lot of issues we’re going through. I hope we can buy you a beer sometime in appreciation.” Thanks for that, Justin. I really appreciate that. Justin is with forekast.com.
If you have feedback or questions for the show, you can do what Justin did and email firstname.lastname@example.org. I read every email that comes into that inbox.
If you haven’t left us a five-star review in whatever podcast app that you listen to, I’d really appreciate it. We’re in Spotify, Downcast, Overcast, iTunes, and all the Apple podcasts. We’re in all the places. Every five-star review that you leave puts a big smile on my face. It also helps keep me motivated to keep doing the show and it helps us to find new listeners. Thanks for tuning in this week. I’ll see you next time.