“Discouragement is an indicator that something is not right:” – Karim Varela
About Coffee Meets Bagel
- Leading mobile dating site which empowers women
- Millions of users with 60,000 concurrent users at peak times
- ~120 AWS instances in use at any given time
- Databases used: Cassandra, Postgres, and Redis
About Karim Varela
- CTO at Coffee Meets Bagel
- Previously built the Android version of Tinder
- Music of choice: David Guetta, dubstep, house, reggae, and some alt rock
- Music of choice while coding: instrumental and jazztronica
- Survived a violent robbery in Costa Rica
We met with Karim Varela, CTO at Coffee Meets Bagel, in their San Francisco office, right off of Market St., and had the opportunity to learn more about the leadership, team, and beliefs that enable Karim’s team to build a consistently high quality product. We covered many things including :
Android vs IOS app. If you’re trying to build a global scale app, you should start on Android (80% of the market share globally). If your goal is start making money from rich Americans, than start on iOS.
On pursuing a career in tech, and how San Francisco is great, but not the only place. Starting off in Portland, Karim worked out of hacker houses in Bali, Costa Rica, as well as working full time in Los Angeles for Tinder before making the move to San Francisco. At Coffee Meets Bagel, Karim manages an active team of engineers both in their offices in San Francisco, as well as remote teams.
How the re-airing of a pitch to Mark Cuban on Shark Tank impacts traffic by 2 to 3 times the normal load.
How building a side project with 10 people (!!!) impacted product quality and marketing, and ultimately affected the project’s ability to get funded. It’s so hard to get focus on delivering really good work while staying on top of all of the pieces together. What separates a hobby from a business is the willingness to sacrifice everything or figuring out how to do it full time.
How collaboration yields great ideas but doesn’t all lead to a better product. What makes the difference is getting your head down and getting real work done. Although collaboration is great the frequency and attendee volume of collaborative meetings can strangle engineering productivity.
How agile software is great in speeding time to market, but doesn’t always lead to the best result. In many cases agile software can be used to justify carelessness and hastily defined product requirements.
We also want behind the scenes on philosophy including objective morality vs the golden rule and how this influences decision making both in professional and privately and when is it time to say “I quit.”
You can follow and reach out to Karim here:
- Twitter – @karimvarela
- Blog
More resources:
- Coffee Meets Bagel Tech Blog
- Coffee Meets Bagel Careers
- One of many 90s style smooth jazz playlists or this new one on Spotify
- Jazztronica on Spotify
- Glass Animals on Spotify
Show notes:
- How Coffee Meets Bagel works 5:55
- Handling traffic from 10K to 60K concurrent users within just a few minutes 07:00
- Empowering the female experience in the dating world 8:00
- Using Amazon Web Services (AWS) auto-scaling groups to scale up resources 9:30
- Impact of the first Shark Tank pushing 10x traffic 11:15
- Running out of memory on Redis and how we solved it 12:45
- Challenges of scaling Postgres by optimizing queries and denormalizing data14:45
- Evolution and revolution of microservices 16:00
- Python vs scala 16:45
- Motivation to join Coffee Meets Bagel as CTO
- The Android vs IOS – how we decided 17:00
- Dealing with the first layoff in his career 17:30
- Making the decision to move to San Francisco 20:00
- Hiring is everything – how we compete with Google and Facebook for the best 21:00
- Dealing with failure 23:00
- What “command and control” leadership is and how it can holdback teams from getting things done. 24:00
- A real example of how empowering leadership by motivating the team to create and make accountability KPIs. 26:00
- Dealing with bad apples / unhappy employees and how empathy can quickly bring out the toughest of issues. 27:00
- How fearless candor, being direct and fearless, helps to get things done 28:00
- Dealing with distraction on how it kills engineering productivity 28:30
- Discouragement is an indicator that something is not right 30:00
- We are a data analytics company that helps people to date 31:00
- How hard it is to start a project on the side of a day job 32:00
- Past experiences building early stage startups 33:00
- We all should be creating and building things on the side, first as hobby, and potentially second as a real business. 35:30
- Android vs IOS decision is really a question on global scalability vs nice Americans. 36:00
- Morning routine and nutrition 37:30
- San Francisco is not the only place to pursue a career in tech 38:00
- Dealing with distraction, the open office dilemma, and working from home 41:00
- Making the decision to work at a resort in Saipan vs getting an engineering job in 2003 42:00
- Battling interviews to get a real first time engineering job 45:00
- How building for websites for SMBs in Saipan motivating Karim to build really cool big things 47:00
- Collaboration yields great ideas but doesn’t all lead to a better product 48:30
- Belief in pair programming 50:00
- Impact that agile software has on hastily defined product requirements 51:00
- Prolific speakers are not always prolific thinkers 52:00
- Objective morality vs the Golden rule.53:00
- Preferred music 55:00
- Knowing when it’s time to say quits 57:00
- How getting mugged and beaten in Costa Rica changed Karim’s life 59:00
- How getting the workout done in the morning adds more inertia for the day 1:01
Full Transcript
Cameron Peron
00:00:00.00
Ok Karim, welcome to the podcast.
Karim Varela
00:00:03.30
Thanks, Cameron, its great to be here.
Cameron Peron
00:00:05.58
Absolutely. Tell us a little bit about Coffee Meets Bagel.
Karim Varela
00:00:08.99
Sure. So Coffee Meets Bagel is a top tier dating app. We have a great reputation for being the dating app that people use to meet their husbands or wives and form really solid relationships. We focus on quality over quantity and we do a lot of work on our algorithm and the data science front to give our users a really curated set of potential matches which we call bagels.
Cameron Peron
00:00:42.41
Great. Great. When did you feel that you were making real progress as a CTO?
Karim Varela
00:00:49.13
I think that there was a point at which I stopped worrying about all the little details, there was a point at which I stopped reviewing every single pour request that came in, and there was a point where I really started to focus on the high level things and, you know, it was that point where I felt like that I could execute on my vision as a CTO, and that we could start moving towards the type of platform, the type of system that we want. I think that was kind of the turning point for me.
Cameron Peron
00:01:26.41
Got it, ok. What did the stack look like?
Karim Varela
00:01:28.43
So we’re pretty much completely on AWS. We have a little bit in Google Cloud. We use a number of AWS services; of course, EC2, S3, SQS, Lambda, Kinesis, RedShift, Cloudfront, Elastic Cache. Our primary application on our backend is written in Python. We use Django framework mostly just for web services. We use a number of different data storers, Cassandra, Postgres, Redis, we use Titan, but we’re getting off Titan, and then we have mobile clients. That’s our only access point to our backend are our native android and iOS clients.
Cameron Peron
00:02:14.07
Ok. How large are the data sets of Cassandra, Postgres, and Redis?
Karim Varela
00:02:16.82
So…Cassandra and Postgres each have a few terabytes in them. We have a 2 cluster Cassandra buildout, we have 5 postgres nodes, a master and four slaves. Redis, I think we have a few hundred gigabytes. We use a few different flavors of Redis, we use like redis cluster, mostly for data that we don’t need to do, set intersections on, that’s easily shardable across, you know, horizontally. We use traditional like master-slave redis. We put data in there that we definitely have to do set intersections on. And we use elastic cache also, the manage redis from AWS to do some other things.
Cameron Peron
00:03:08.88
How do you decide between using an open source solution like redis and having resources managing that versus using the manage service like elastic cache or something else?
Karim Varela
00:03:21.14
Yeah, so, a couple different ways. Number one it comes down to what we have familiarity with. I have an awesome devops team and if we’re looking at a technology that we haven’t used in our stack before and we have experience with it in-house and we don’t think we need to pay for a managed service, then we’re going to try to do it in-house. If it’s something that we’re not too comfortable with and it’s not too expensive, then we’ll go with the managed service. But it’s a fluid decision making process that depends on the technology involved and the expertise that we have in-house.
Cameron Peron
00:03:54.34
Ok. What other open source technologies are you currently using?
Karim Varela
00:03:59.10
Pretty much everything we use is open source. Python, django, is open source, our android app is pretty much all open source. All of our data storers – cassandra, postgres, redis, all open source. We use a number of different things that kind of plug in to python, like gunicorn, gevent, these are opensource tools to help us with concurrency on python because concurrency on python kind of sucks. We use ngenx for routing at the webserver level, to our different web services. I forgot to mention earlier, we’re kind of starting our evolution of the platform into microservices, and a lot of new services that we’re starting to build out are in scala. We also used golang for some devopsy things. So python, scala, and golang, all open source languages.
Cameron Peron
00:04:58.46
What are some of the craziest backend challenges that you’ve faced to date?
Cameron Peron
00:05:02.22
Actually, sorry, before we get to that, give us a sense of what kind of scale you’re trying to meet.
Karim Varela
00:05:07.65
Ok, sure. So I can’t talk about specific user numbers but we have millions of users. We have, if you look at our peak traffic it goes up to around 60k requests per minute. So that’ll give you a sense of the kind of scale that we have to handle at our peaks.
Cameron Peron
00:05:30.69
Right. Ok. Good. If you could describe some of the craziest backend challenges, like specifically that you’ve had to face – what it was and what you did to solve them, it’d be great.
Karim Varela
00:05:45.00
ok, sure. We have a crazy backend challenge every single day at noon. If you’re familiar with our product, our gimmick is that every day at noon we send you your batch of bagels. And what that means is – we don’t actually send you anything – we send all of our users a push notification at noon in their timezone. So it starts at noon eastern time, that’s our first slam of traffic. So right around noon eastern time, we send all of our users on the east coast a push notification and right around then, when they get the push notification, they’re having lunch, they’re like, ok, cool, my bagels are here. I’m going to open Coffee Meets Bagel. They open the app and we get flooded with network requests. So literally our requests per minute go from 10 or 20k per minute to 60k per minute within like a couple of minutes. So it’s like this violent surge of traffic and that is a challenge, that is a constant challenge for us, to make sure that our systems are warmed up, to make sure that our systems are pre-scaled out to handle that traffic, and to make sure that our database layer is going to be able to handle all the requests coming in to it.
Cameron Peron
00:07:02.49
Right. So all day long, the end user process or flow is one where guys and girls are like swiping left to right if they like each other…and then?
Karim Varela
00:07:11.57
So we’re not a swiping app. There’s no swiping in our app. We have buttons for liking and passing and that’s because we want you to take a little bit more time to evaluate your match. Sorry to interrupt, I just wanted to make that a point, that we’re not a swiping app.
Cameron Peron
00:07:30.00
Right, and it works in a way where the woman chooses, right? Isn’t that how it works? Explain it, please.
Karim Varela
00:07:42.55
Yeah, so, our company is founded by three sisters, the Kang sisters, and our mission has always been about empowering the female experience in the dating world. And so a lot of our features are geared towards making the female experience better, safer, easier, less intimidating, what have you. So, in the primary flow of our app, men see a lot more potential matches than women. Because men are fine with going through many more matches than women. So men get maybe 20 potential matches in a day and they choose which women that they like. And then the women, for the most part, only see men who have already liked them. So for a woman, the experience is really streamlined. Pretty much every single guy that she sees has already liked her and she can just pick the guys that she likes. Those guys will most likely meet the majority of her criteria so we’re going to prioritize showing the woman guys who have already liked her, who meet probably all of her criteria, hopefully.
Cameron Peron
00:09:01.44
So, you’re saying this is a major challenge, so at noon you have around 60k requests suddenly, and it happens a few times across the United States and I’m sure it happens in different countries as well. So what does that look like and what have you done to meet that?
Karim Varela
00:09:16.88
Sure. So what we do is we use on AWS we use autoscaling groups and you can schedule them on AWS to scale up at certain times of day so before noon on the east coast we scale up our webservers dramatically in order to handle that traffic. That’s easy enough, it’s easy enough to scale up your webservers so that you have enough free processes to handle all the requests coming in. That doesn’t necessarily mean that your database layer is going to be able to handle all those requests. So we kind of have to make sure that our database layer is always scaled up, because you can’t scale up and down your database layer on demand, really, because there’s no way to like copy all the data that you need to into a new database in time to like scale up within three minutes for a violent traffic surge. So we have to constantly monitor our databases to make sure that they’re going to be able to handle whatever the next traffic spike is. So we have to forecast our traffic out and know that these violent traffic spikes, they could be 3x they could be 5x. If we do a marketing campaign or our shark tank episode has aired again, it could be 10x. So it’s a little bit of an unknown and a little bit of an art that we’re constantly perfecting all the time.
Cameron Peron
00:10:44.00
What was the impact of the shark tank episode on your stack?
Karim Varela
00:10:47.27
Well, the very first time it aired, I was actually not CTO then. But it was a huge business effort to prewarm our systems and to prescale our database layer so that we could handle that traffic. I think it was something like 10x. But our shark tank episode was one of the most popular because Mar Cubin made like this huge offer to basically buy the entire company, so they keep re airing it, like every couple of months they re air it and we get spikes in traffic and we don’t know when it’s going to happen. So it’s just some random evening every couple of months, or sometimes they’ll like air it in Australia and it’ll be in the middle of the morning or something, and so we don’t know when it’s going to happen and that could instantly increase our traffic by 2 or 3x.
Cameron Peron
00:11:44.16
Ok, got it. We were speaking earlier about running out of memory in redis. What happened exactly?
Karim Varela
00:11:53.56
Well. When you add too many things to redis then you run out of memory. And when redis runs out of memory then it crashes. And if you don’t have your data backed up, then you lose data. And so we had a bad outage around thanksgiving because we made some application changes and we started adding more data than we thought we were adding to redis and the memory just kind of exploded and, unfortunately, we had turned off backups on that cluster because we were seeing this weird behavior where every time a backup would happen, even though it was supposedly happening in the background, we’d see this latency spike in redis and that was not acceptable for us. So we turned off backups and just kind of forgot about it. We got caught up in day to day work and the next thing we know, redis is crashing and our last backup is like three months old. So we basically lost three months of data within a split second and we had to – luckily the majority of that data, I think like 90% of that data, was permanently stored in other data storers, so we were able to recover it and we were able to get our system back to a working state within a few hours and we were able to backfill all the data within a few days. But our users had a degraded experience for a few days. That was one of our worst outages ever.
Cameron Peron
00:13:31.31
What did you do to resolve the problem exactly?
Karim Varela
00:13:37.16
So we, well first, we inspected the recent pour requests, we figured out what was the recent code change that caused more data to be put into redis. And then we created a new redis slave, whose only purpose was backups. All it did was just sync data from master and do backups. And having that relieved the rest of the cluster so that there was no latency spikes when that particular node was doing backups and so that’s how we got around that.
Cameron Peron
00:14:16.90
We also spoke about scaling your database layers from like a few hundred gigabytes to terabytes. Can you explain a little more about that?
Karim Varela
00:14:24.78
Yeah, I mean since I’ve been CTO, I think that our DAUs and MAUs have doubled or tripled and that means that we’re storing more and more data in our primary, mostly our primary data storer. Postgres is definitely the hardest to scale. The first instinct I think is to scale it up vertically and just use bigger instance types in AWS, so we’ve done that a couple of times, just gone to instances with more storage, with more iops, with more cpus so it can handle more load, but that only goes so far. You can’t rely on AWS to continually create bigger and beefier instance types forever. So a lot of the optimization that we’ve done around postgres has involved optimizing queries, denormalizing data so that we don’t have these queries that are just using up tons of resources.
Cameron Peron
00:15:28.04
Let’s talk about microservices and what your philosophy is on that and how you use them.
Karim Varela
00:15:38.07
So we are just starting in our evolution or revolution to microservices. I look at it as a necessary evolution of our platform. I think that as you scale, you just kind of outgrow your databases and for us, microservices is a way for us to scale our platform I think infinitely and be able to scale up and down individual parts of it in response to load on individual parts of our system, as well as, I think in the future, right now we really only have one app, coffee meets bagel, but I think there’s a good chance in the future that we want to build other apps, maybe in the dating space, maybe in the social space, but there’s a good chance that those apps could use those same microservices that we have for our main coffee meets bagel app, so there’s an element of reusability as well. Another nice thing about microservices for us is that right now we’re a primarily python shop but we don’t necessarily think that python is THE language that we want to go with for the future so having microservices allows us to experiment with the languages, as I mentioned before, a lot of new services that we’re working on we’re building out in scala, for performance reasons, mostly concurrency. Scala handles concurrency a lot better than python and it allows us to be polyglot, it allows us to be poly-datastore, I don’t know if that’s a word, but it allows us some experimentation and we’re all about that. We believe in using the best tools for the job and I think that takes a little bit of experimentation.
Cameron Peron
00:17:31.24
Let’s go back a bit. What was your motivation to join Coffee Meets Bagel as CTO?
Karim Varela
00:17:36.66
So I actually joined as the head of Android. They hired me to build the Android app and to build up the Android team. that’s what I had been doing previously in my career – oh and also I was at a point in my career where I had just got laid off from my last company – that was Tinder – and I got a nice severance, first time in my career where I really got a severance and I had a couple months to figure out what I wanted to do. And it was always my dream to find a company that would let me work from anywhere in the world. So the only companies I talked to were those that let me do so.
Cameron Peron
00:18:22.28
Sorry – were you based in LA or ?
Karim Varela
00:18:27.67
I was living in LA at the time and CMB was one of a handful of companies that was going to let me do that and I picked CMB because of, one: great product, really beautiful engaging product and I liked their – our – mission of empowering the female experience on the dating app. I thought that that was a great idea and I think key to winning in this market. Also I was really impressed with the founders and the executive team. They’re really data focused, we make all of our decisions pretty much on data, and a-b testing, and analyzing our users’ behavior. So I felt this company had the right mix of product and leadership that I wanted to join. And so I joined and I worked remotely all over the world for a couple years and then our previous CTO decided that he wanted to move on and move to New York and build something new, so they offered me the job but they were like, if you want the job, you have to come back to San Francisco and work from the office. That was the deal. But I’m happy to be in San Francisco.
Cameron Peron
00:19:47.69
Why would you move? I mean, there’s a pretty hot scene in LA. What was – was there a personal motivation to move back to San Francisco aside from the job as well? There’s a high cost of living here, have to move again, make new friends….Significant other may or may not be happy about that, was there another kind of motivation around that?
Karim Varela
00:20:11.52
For me – I don’t really get grounded in places, I’m kind of nomadic. I kind of just chase the work and I’m passionate about the work, and so for me, it’s more important for me to be passionate about the work than to be somewhere where I’ve made friends or that’s warm – like LA. LA is nice. I do miss the weather there, and the beach…but…(and the traffic right?)…well I ride a motorcycle so traffic’s not a problem. But for me, I’d rather join a company that I’m super excited about rather than just stay somewhere because I’m there.
Cameron Peron
00:20:56.13
What are the mistakes from Tinder that you learned?
Karim Varela
00:21:01.07
Oh geez…I think the…mistakes from Tinder….
Cameron Peron
00:21:13.34
We can come back to it if you want to think about it, that’s ok.
Karim Varela
00:21:16.65
Ok, let’s come back to it.
Cameron Peron
00:21:18.79
Ok, let’s talk about how you hire engineers. How important is hiring to you, and how do you find and convince talent to join the team?
Karim Varela
00:21:33.49
Hiring is everything. We have a huge emphasis on hiring here. We pride ourselves in hiring world class engineers and every single one of my engineers out there is amazing. We have a few primary avenues of recruiting, one, we have an inhouse recruiter, he’s our head of operations and he reaches out mostly on Linkdin. We also have a tech blog tech.coffeemeetsbagel.com where we highlight our engineering work and hope to recruit people that way and some of the senior engineering and leaders here, we go out, go to meetups, conferences, and try to meet people. But it’s always a struggle. We’re not google or facebook, but I think that we offer things that those big companies cannot, like the ability to wear many hats and perform many job functions and have access to the CEO and the CTO and you’ll be working with those kind of people on a day to day basis and we just have a really closeknit group and I think really there’s a lot to be said for working for a 35 person startup versus google or facebook or something like that.
Cameron Peron
00:23:04.68
Can you give us an example of how you failed as a CTO? Failure is really important in any company.
Karim Varela
00:23:12.13
I think failure is important. I think for me some of my failure initially was in having this kind of like command and control attitude about the job. I came in and I had strong opinions about, especially process wise, like what’s the best way to run teams, what’s the best way to build software in agile fashion and I think I tried to shove those positions down people’s throat a little bit and what I’ve kind of learned as a leader is that it’s a lot better to express your vision clearly and concisely and try to get buy-in for the vision, kind of explain where you want the company to be at but not necessarily how you want your leaders to get there.
Karim Varela
00:24:18.65
So that was definitely a maturity that I went through as a CTO.
Cameron Peron
00:24:24.71
How….Can you give an example of how that actually looked?
Karim Varela
00:24:28.66
Yeah…I mean like the biggest example that I think some of my engineering managers struggled with was, you know, when I came in, like, some of our teams were tracking velocity per sprint, some weren’t, and I wanted everybody to track velocity because I wanted to be a little, I wanted us to be able to determine and analyze how we were improving or not improving as an engineering organization. So I kind of came with a plan, I tried to get buy-in on the plan, there was some pushback from a couple of my senior engineers, leaders of engineering teams. And it turned into kind of like a nasty argument because they didn’t want to be told how to run their engineering teams. So if I had to do that over again, I would say, “Look, it is our goal to be able to track your velocity because this is how we can measure our performance as an engineering team. How you do that is up to you, this may be my recommendation, but I’m not going to dictate to you how to do that.”
Cameron Peron
00:25:40.25
Great. Ok, so you’re empowering your tech leaders to generate their own KPIs with their teams?
Karim Varela
00:25:49.98
Yeah, for sure. This was specifically around how much work they’re getting done in a sprint, but yeah, I also, and us as a company, we try to get our engineering leaders to come up with their own KPIs, the things that they think are important for their team to be hitting, and we have them report on those KPIs at our all hands meeting.
Cameron Peron
00:26:15.72
How do you deal with bad apples on the team?
Karim Varela
00:26:19.19
Well…
Cameron Peron
00:26:21.33
Disgruntled, exhausted, stressed out guy that’s just bringing in a negative vibe to the project maybe, just not bringing you into the right kind of attitude, maybe…sabotaging other people’s work without really even knowing it? Have you encountered those kind of instances before in the past?
Karim Varela
00:26:40.65
Not too much, but a little bit here and there.
Cameron Peron
00:26:44.19
In a small team it can be totally destructive, right, because it can be so subtle.
Karim Varela
00:26:48.57
Agreed. From my experience, you really have to get down to the root of the issue and find out why someone is disgruntled – do they feel like they’re not being heard, do they feel like their opinions don’t matter, which I think is a lot of time the case, if somebody feels like they’re not being included in processes. Do they feel undercompensated? It’s important to get down to the bottom of it and have a candid conversation and one of our company values is “fearless candor” so we actually look for that in people we interview.
Cameron Peron
00:27:32.42
Fearless candor? What does that mean?
Karim Varela
00:27:35.05
That means being forthright in your conversations, not ever being afraid to voice your opinion, and being able to listen to someone’s feedback about you without taking it personally, and understanding that the reason that we’re candid with each other is for the benefit of the company.
Cameron Peron
00:27:57.90
I like that. That’s great.
Cameron Peron
00:28:04.77
How do you deal with distraction or with discouragement? Both in your personal life and in your business life?
Karim Varela
00:28:11.77
I think those are two very different things, so in terms of distraction I think we all know at this point in our engineering careers that distraction kills engineering productivity. I think it was microsoft who did a study on it and they determined that like every time you distract an engineer, if they’re deep into solving some problem, then it could take anywhere from 5 to 15 minutes for that engineer to get back into their problem. So reducing and eliminating distraction has actually been a huge priority for me. I encourage my engineers to block off, turn off all their notifications, whether that be on their phone (especially on their phone), on slack or email. You know, if you need a two hour block or more of time to focus on a specific problem, then do it. Nobody’s going to hold it against you that you didn’t reply to your slack message within five minutes or whatever. I also encourage people to only check their email a few times a day – maybe when you come in, maybe after lunch, maybe before you go home for the day. And also, just if you need to, or maybe even if you don’t need to but if it helps you to just block off time on your calendar for times that you want to be uninterrupted. So that’s important.
Karim Varela
00:29:43.47
In terms of discouragement I think that again, like a disgruntled employee, discouragement is an indicator that something is not right. Something is wrong, and so we need to have a conversation about it, we need to be candid about it. I hope that discouragement is not happening too often here. We have a culture of continuous learning here so all of our engineering teams, after every sprint, they have “post mortem” and they talk about what they did well, they also talk about things that went wrong and what we can learn from them. So I would hope that any discouragement is not lasting more than a sprint. And if it is, regardless if it is or not, we have to find out how to make it better. That usually involves experimentation, trying new things, getting creative, thinking outside the box, so we can figure out how to make it better.
Cameron Peron
00:30:44.10
You mentioned earlier about using data to be more competitive. So explain more about how you look at, like, competition and how you use data to be more competitive.
Karim Varela
00:30:57.40
This is how I look at CMB. We are a data analytics company that happens to help people date. That’s not in our marketing message but if you look at our day to day operations, a lot of it centers around data. So we’re constantly gathering data, we’re constantly analyzing data. We have an awesome data science team here that is constantly experimenting with our algorithm, constantly pulling in new data sources to figure out what data source matters. What are the points of data that actually make a difference in the dating business? And I think that is our competitive advantage and that’s why we have some of the highest connection rates in the industry, and that’s why we’re able to match people up so effectively and we have so many marriages come in all the time.
Cameron Peron
00:31:58.30
For an engineer that is thinking about developing an application now – a few questions on this. One is – How do you focus on doing that without exposing yourself to too much risk, in other words, does it make sense to start a project on the side and then scale it up while still working in a company or to find a certain like test of scale where you meet certain parameters and then go full on into it, manage multiple projects at the same time, see which one works? Which one, in your opinion which one worked the best either for you or that you really believe in?
Karim Varela
00:32:35.98
So I have tried to start a few companies on the side, to varying degrees of success, and it’s really frickin hard.
Cameron Peron
00:32:46.04
story of my life…
Karim Varela
00:32:49.80
I’ve started three different companies nights and weekends while having a day job. And it’s exhausting. You feel like you never have any time to yourself and in my opinion, your work suffers. You’re not able to do your best work on your side project and you’re not able to do your best work at your day job.
Cameron Peron
00:33:16.10
What was a project specifically that you’re thinking about….
Karim Varela
00:33:22.70
The last one was called social tag. We built a service that, I like to say, it was a combination of eventbrite and linkdin. It was event networking software, so we had mobile apps, we had a backend platform. You go to an event, you fire up our app and it would tell you who else is at the event and give you kind of a leg up on the networking. Built up a team of like ten people in LA, we’re all working for equity, all working part time, nights and weekends for the most part, and ultimately we were never able to make the product good enough and to put enough effort into marketing to get funded. So that company never got funded. Before that, in Seattle, I started a company called ibooze and it was kind of one of the first on demand delivery services. This was before like the iphone and android got big, so it was a webservice, and you go on, you order, mostly alcohol, and we deliver.
Cameron Peron
00:34:32.70
Sounds very legit. Problem free, right?
Karim Varela
00:34:38.08
We deliver it to you within an hour and it was actually going very well. It was a profitable business, I was running. I started it in my garage, stereotypical startup in the garage, outgrew my garage, got a warehouse in downtown Seattle, had like 4 full time employees, while I was still – it never got big enough for me to quit my day job, but it got big enough to sustain itself. But I was exhausted, I would finish my day job at 5, go run ibooze from like 6 till 2 in the morning, rinse and repeat. And as a result of that, I overlooked some legal issues, and the city of Seattle ended up shutting the business down. So I lost that business – I thought that was going to be my ticket to early retirement. It’s hard to stay on top of all the different pieces required to run a successful business. Now, if you’re just talking about a hobby, sure, we should all have hobbies, you should be building stuff on the side, you should be experimenting, you should be going to hack-a-thons and learning and building and creating – if that’s your thing. It’s definitely my thing and so I always like to be creating but at the point at which you want to start a business, you better be willing to sacrifice everything or you better be willing to figure out how to do that full time or else it’s going to be really hard.
Cameron Peron
00:36:12.52
When does it make sense to invest into building an android application?
Karim Varela
00:36:22.52
That’s a loaded question. So I came from Android, so I’m probably biased here, but I was an Android developer for like five years. So I don’t think there’s ever not a time to build an Android app. Unless your market is just like rich Americans. Like if rich Americans are your market, which like in San Francisco, everyone thinks rich Americans are their market. So sure, go build your ios app first. But if you’re trying to build a global scale app and a service that’s on a global scale, in my opinion, you should start on Android because last time I checked – it was probably a year or two ago – but last time I checked, Android still had like 80% market share globally. So if your goal is to get users and to build up a huge platform, I would start on Android. If your goal is to make money off rich Americans, sure, start on ios.
Cameron Peron
00:37:26.60
Let’s talk more about you. So how does your morning look like when you wake up?
Karim Varela
00:37:34.11
I wake up at 6. I go to the gym or go on a run, then I feed, and get to work around 8:30 and start cranking away.
Cameron Peron
00:37:42.05
How about breakfast?
Karim Varela
00:37:43.24
Breakfast is a protein bar before my workout and then a smoothie afterwards.
Cameron Peron
00:37:51.29
What is your opinion on living in San Francisco? Is it a place that one should live if they’re pursuing a career in tech?
Karim Varela
00:38:03.06
I think San Francisco is a great place to live if you’re pursuing a career in tech. But it’s not the only place to live if you’re pursuing a career in tech. I think that if you’re pursuing a career in tech you should do it from wherever you are and if the job, whatever job, whatever company that you want to work for happens to take you to San Francisco, which it has a few times in my life, then take it. SF is great, it’s a great place to live. SF is great because you have this incredible community of entrepreneurs and technologists here, you have a ton of smart people. But there are pockets of smart people all around the world. I lived in Indonesia, in Bali, and there’s hacker houses in Bali. There’s great communities in other places around the US, Austin, New York, LA, Seattle, Portland is a burgeoning tech scene. I think it’s….I wouldn’t recommend just moving to SF because you think that SF is going to launch your career and get you into tech and it’s going to be so great. I know a lot of people who’ve moved to SF and they’ve been disappointed because it’s really hard to get a job here. It’s really competitive. So if you’re not the top of your field, it’s not easy. There are a lot of jobs here but there’s also a lot of engineers here trying to get jobs. I think that quality of life matters and you might make more money in SF but that doesn’t necessarily mean your quality of life is going to be better because everything costs more in SF.
Cameron Peron
00:39:45.72
But isn’t it relative? Like if you were to live in, I don’t know, like Omaha for example. It’s all relative so you might be spending less, but your ability to earn is much less too. And even if you’re working as an outsourced engineer, your salary is adjusted, right?
Karim Varela
00:40:04.51
Sometimes. Sometimes not. Some companies definitely adjust your salary, some don’t. I’ve worked remotely and I’ve made pretty close to what I would make in SF. We have people working remotely. We have someone in Seattle, we have someone in Missouri, and they make pretty close to what they’d make in San Francisco. And that’s a practice for a lot of companies, so, I think you should stay put, but if the job takes you to SF, or if the job takes you to New York or LA or Seattle and you’re passionate about it, then you should follow your passion.
Cameron Peron
00:40:47.46
A question on that is managing teams that are working remotely or in an open office – how does that impact productivity of your team?
Karim Varela
00:40:59.23
It varies. A lot of people are (myself included) more productive outside of the office. So if you can lock me in a hotel room or wherever – in my house – I’m going to be more productive than I am at the office because I get distracted at an office. Other people, they get distracted at home because they have kids or the tv is so close or they take a nap or whatever. So it’s all on an individual basis and for us, we only hire remote people who are very senior, at the top of their fields, people that we know we can trust. For the most part, everybody else comes to work at the office everyday, although we do have a pretty flexible work from home policy because we want people to be able to have the flexibility in their lives to work when they’re at home and also we know that, like, people have other things going on in their life. So when they’re here, even when they’re here, they’re going to have to do personal things sometimes and we know that when they’re home they’re going to be working sometimes. So we try to have a flexible environment that caters to people’s needs.
Cameron Peron
00:42:17.99
Let’s talk about your own career – as an engineer. You graduated college in 2003, right (yep). What happened – there’s a recession, post recession, pre 2008, of course. Did you go right into a company? Did you take some time off, what did you do?
Karim Varela
00:42:40.56
I graduated in June of 2003 and the economy was still reeling from the dotcom bust. It was tough to get a job but…there…I went to UC Santa Barbara, there were, I studied computer science there) there were recruiters who’d come there from companies. So I talked to some and I got one job in software. It was building software for a bank, I think it was in java, which was what they taught us in school back then. I hear java’s making a comeback, by the way. Anyway. So that was one option. The other option was this recruiter came from this resort on Saipan called The Pacific Islands Club in Saipan.
Cameron Peron
00:43:33.15
Hold on, so everyone knows where this place is – Saipan is like in the middle of the Pacific Ocean right? like south of Hawaii??
Karim Varela
00:43:42.10
It’s actually much farther west of Hawaii. It’s actually south of Japan. It’s in the middle of the South Pacific in a group of islands called the Northern Marioness islands which are all owned by the United States. The US took them in WWII from Japan. You have Saipan, Tinian, Guam, Rota, I might be missing one. Pretty small islands, I think this island Saipan was six miles by ten miles. Anyway, this recruiter came from this resort and he’s looking for windsurfing instructors and ocean lifeguards and swimming instructors, and I happened to be perfect for the job because I was a collegiate swimmer and I was also an ocean lifeguard and so they offered me the job. So I had two choices and I was like software can wait so I took this job out in Saipan and I lived there for about a year. It was paradise and I don’t regret it, but when I came back it was even harder than I expected it to be to find a job in software. I was 23 years old, I had never done an internship or anything in school because I was ocean lifeguarding and just having fun and I interviewed with a lot of companies and nobody would hire me. I probably interviewed with like 20 companies. And finally I ended up getting a job with Electronic Arts down in Redwood Shores, CA. And my job, it was called a language integration technician. Basically the job was to take files of strings, string assets in other languages like german, spanish whatever, language didn’t matter, and put it into games – both console games and like handheld games like gameboy and stuff and just make sure it worked. So I put the language assets in there and played the game, make sure everything looks right, all the sounds are playing, everything. So it was kind of like tedious work but I did get to play video games all day, but I don’t know, for anyone who’s ever tested video games, it’s not that fun. The video games are like in varying states of completion from like just starting out to like half baked to nearly done and you have to play the same game over and over and over again. So anyway, they laid me off after like four months. Yeah Electronic Arts has a nasty reputation for that. But it was a blessing because I got into mobile and I got into mobile before smartphones and I’ve never looked back. Mobile has been a blessing for my career.
Cameron Peron
00:46:27.89
If you were to start it all over again, to the day that you decided to be a software engineer, what would you do differently? If you were to go back like to when you were thinking about going to Saipan, specifically, what would you do differently?
Karim Varela
00:46:43.50
Oh. I would’ve still gone to Saipan. But I would’ve focused on building something cool while I was there, in my spare time. I was kind of like, at that point of my life, I was like oh it’d be cool to have a boutique web development shop so I was taking small jobs here and there, building websites for just random small businesses and people. And making small money here and there but it really didn’t move the needle. But I thought that was a good use of my skills. And if I look back on it, at that point I would’ve focused on building something that I’ll want to use, I would’ve focused on building a startup, building something cool. Those kind of thoughts just weren’t kind of in my head at that time. It wasn’t until I got into like my mid to late 20s that that kind of registered as that’s what I want to do with my career is I want to build cool things.
Cameron Peron
00:47:48.23
A few more questions. What is one opinion that you hold very close that others wouldn’t agree with that specifically pertains to your work environment and business?
Karim Varela
00:48:02.56
I think…so in terms of business…one is that collaboration doesn’t always lead to better product or more productivity.
Karim Varela
00:48:22.64
I think especially with this millennial culture we have, they’re very focused on collaboration. A lot of new young employees I get in, they’re all we want to collaborate, we want to collaborate, we want to collaborate. And I think that’s great, I think collaboration yield a lot of great ideas, but there has to be like — to be an effective engineering organization, and collaboration is great in engineering, don’t get me wrong. But there is a point at which in order to be the most productive, in my opinion, you have to kind of stop collaborating and just kind of put your head down and get some work done.
Cameron Peron
00:49:05.88
Does that mean that collaboration – is it like a degree of like how many people or collaborating together and the result of that or is it just like the principle of collaborating too frequently?
Karim Varela
00:49:18.34
It’s both. I think that we, in a lot of startup culture, we kind of take it too far. We’re pulling engineers into meetings all the time to get their opinion on things and it’s a distraction. And so I think we need to be very careful about how much and how often we’re asking our engineers to collaborate.
Karim Varela
00:49:47.06
But, I just want to make it a point, I do believe in pair programming. I do believe in pair programming on a regular basis. So it’s kind of a distinction between collaborating with your engineering peers versus collaborating with design, product, marketing, etc. I think the job of a lead engineer or a manager is to kind of shield their engineers from those kind of collaborative meetings.
Cameron Peron
00:50:17.48
Any other opinion that you have business wise you’d like to speak to?
Karim Varela
00:50:23.49
I think another thing that we’ve kind of gone overboard with lately is agile. I think the agile is great, we should strive to be agile in the way that we build software, but it doesn’t always lead to the best result. Especially if we’re often changing course midway through projects or requirements are not explicitly defined or clearly enough defined. I think that gets us into a lot of trouble and I think a lot of companies, a lot of product people, they kind of fall into this crutch of “oh, we’re agile, so that means I don’t have to like clearly define this product” or “we’re agile so that means that we don’t have to write product requirements.” I think that’s a trap that leads to number one, it takes longer to build something and number two, it ends up being more buggy because engineers aren’t focused on building the best thing right at the outset.
Cameron Peron
00:51:26.63
How does that opinion apply to your own personal beliefs?
Cameron Peron
00:51:34.62
Sorry. Tell me an opinion that you hold closely to yourself that others may not agree with that is on a totally personal level, not connected with work.
Karim Varela
00:51:44.23
So one thing that I believe is that prolific speakers, or people who talk a lot aren’t always the prolific thinkers.
Cameron Peron
00:51:54.78
Who, specifically?
Karim Varela
00:51:56.51
Anybody.
Cameron Peron
00:51:57.49
But seriously, like who? Like….a real example?
Karim Varela
00:52:01.78
A real example…
Karim Varela
00:52:04.39
I mean….I have friends, I’m not gonna name people. But I think there’s some people who really like to hear themselves being heard. And they like to hear themselves speak, I should say. And there’s a lot of people who have really valuable opinions and really valuable thoughts but they’re not so talkative and so we don’t hear them. And I think that that’s a problem and I think that as leaders we need to kind of reach in – reach out to those people and get those thoughts out of them.
Cameron Peron
00:52:48.23
Right. How do you see that happening? Or what have you done to enable that?
Karim Varela
00:52:55.34
I think in a business setting it’s all about being inclusive. After you’ve worked with people for some amount of time, you know who are the ones who never hesitate to share their opinion and you know who the ones are who do hesitate to share their opinion and we hire everybody for a reason and, like I said, I have a lot of amazing engineers. They’re all super smart, every single one of them. But some of them like to talk more than others, and the ones who don’t like to talk so much…it’s I think it’s our job as leaders to specifically ask for their opinion, because otherwise you might not get it and you could be missing out on valuable information.
Cameron Peron
00:53:40.91
Is morality subjective?
Karim Varela
00:53:47.07
Yeah, I think morality is subjective. I don’t necessarily believe in morality. I believe in the Golden Rule. I think that we all have our own version of what morality is, right? Morality is different for every single person. It’s different as humans have evolved. Certainly like cavemen had a much different idea of what was moral and immoral versus what we have. Certainly even our ancestors 200 years ago who had slaves had a very different idea of what is moral and immoral compared to what we have. Certainly even 100 years ago, when we didn’t allow women to vote, we had a different idea of what is moral and immoral. So I think that morality is a constantly evolving thing and there’s no concrete this is moral and this is immoral.
Cameron Peron
00:54:46.89
Hard to ask you another question after that one.
Cameron Peron
00:54:54.76
We’ll go a different route. What kind of music do you like to listen to and specifically what do you listen to when you’re coding.
Karim Varela
00:55:06.04
I like a lot of reggae, I like a lot of electronic music, alternative rock, some hiphop. When I’m coding, I don’t like to listen to music with any words so all instrumental…
Cameron Peron
00:55:19.55
What kind?
Karim Varela
00:55:21.08
Well, electronic music. Maybe dubstep, maybe house, maybe trance. Also I’ve got some jazz – smooth jazz especially.
Cameron Peron
00:55:28.45
1990s style smooth jazz?
Karim Varela
00:55:29.96
Oh yeah.
Cameron Peron
00:55:31.20
Love that stuff. it’s disappeared, kind of like you have to find old 1990s stuff on youtube now. Smooth jazz.
Karim Varela
00:55:37.35
I recently got into jazztronica which as you might guess is (c: also good, spotify has a good channel on that, hiphop jazz is good too.)
Karim Varela
00:55:52.69
The next show I’m going to is The Glass Animals. Super stoked about that. Definitely recommend them if you haven’t heard of them.
Cameron Peron
00:56:02.26
We’ll put it in the show notes. Who else specifically do you like to listen to in those genres? Any names come to mind?
Karim Varela
00:56:14.20
I can think of shows that I’m going to. I’m going to glass animals soon. After that, I’m going to Odessa, that’s kind of like trance-y electronic music. Like dubstep – I like glitchmob, bassnectar…in terms of reggae I really like hebron gate, black ghuru, some of the more classic stuff, bob marley of course….yeah…a mix of things.
Cameron Peron
00:56:48.34
When is it time to say quits?
Karim Varela
00:56:53.46
That’s a very personal question. For me, I don’t think that I’m ever going to quit building things. I think as long as I live I’ll always have this desire to create things. And that’s my goal, is to always be creating. I think… But hopefully, CMB will be the last “job” I ever have.
Cameron Peron
00:57:19.46
Do you think it will be the last job you’ll ever have?
Karim Varela
00:57:22.05
I think so.
Karim Varela
00:57:23.52
I don’t know, it’s hard to say, like, actually since I’ve been in the CTO role, as you can imagine you get a lot of inbound recruitment from really interesting companies. So it’s hard to say. Like I said, I follow my passion and if I’m not working at CMB sometime in the future and something really interesting comes my way, then sure I’ll take it, but me – my dream is to finish CMB whenever that happens, move to Costa Rica, get a casitas on the beach, surf everyday and build my things.
Cameron Peron
00:58:07.04
What terrifying event happened to you in the past few years that had the most impact in your life?
Karim Varela
00:58:20.18
Hmm. I can think of a couple terrifying events. About a year ago we had a company outing, we played bubble soccer and I blew out my knee. I just destroyed like every ligament in my knee and so i couldn’t (c: that’s great for a company outing) I couldn’t walk for like three months, I was a little worried it would never be the same. I’m still recovering, still going to physical therapy.
Karim Varela
00:58:51.94
Couple years ago when I was traveling, I was actually in Costa Rica and I got mugged and beaten. Pretty badly by three guys. That was kind of terrifying.
Cameron Peron
00:59:05.51
Mugged and beaten. In that order?
Karim Varela
00:59:08.66
Well it was…it kind of all happened really fast. I was riding my bicycle down like a dirt road at night and going back to my apartment and three guys just jumped out of the bushes. They had masks on, bandannas, and they attacked me. I think that – I was working at this coworking space in Puerto Viejo which is on the Caribbean coast of Costa Rica and I had my nice laptop, my cellphones, my ipad. So they probably saw me with all these things, they knew I had them in my backpack and they jumped me. They tried to take all my things. They didn’t have weapons so I tried to fight them and they beat me pretty good but they weren’t able to get my things. But yeah that was pretty terrifying.
Cameron Peron
01:00:01.02
What did that do to your life? Like how do you, what did you learn from that exactly? How did it change you?
Karim Varela
01:00:07.33
Right after that I was…that was on the caribbean side of Costa Rica. The next day I moved to the Pacific side, which is only like a 2 hour drive away, and I think it’s a harsh wake up call into the divide that there is between what we have and what other people don’t have. And it was a little bit of a wake up call and I think more than anything it just made me grateful for the opportunity that I had to travel around the world as a software engineer and to have all the things that I want and to live in paradise. Hopefully made me a little bit more gracious.
Cameron Peron
01:00:53.75
What kind of hobbies do you have? Outside of work?
Karim Varela
01:00:58.31
So right now I’m big into running, I got into running after I hurt my knee. I was like okay, I’ve always wanted to run a marathon so I’m training for a marathon right now, the Portland Marathon up in Portland, Oregon. Recently ran the San Francisco half a couple weeks ago. So that’s my thing right now. I also salsa dance, once a week or so. Before I hurt my knee I was into Brazilian JuJitsu. That’s probably my favorite activity to do but my doctor’s not allowing me to do that until my knee is a little better.
Cameron Peron
01:01:32.78
That would explain how you were able to at least compete with three muggers, i guess, right?
Karim Varela
01:01:37.47
Maybe. Maybe. I mean they were young guys. They were probably late teens, early 20s and I mostly just covered up and took the beating, it wasn’t like I hurt them.
Cameron Peron
01:01:54.82
How do you make time for these activities? Must be a lot of work that you’re managing here. How do you do it?
Karim Varela
01:02:04.58
Wake up early. Try not to go to bed too late. I believe in getting my workout in before work. I think if you get your workout in before work then you have plenty of time to either work late or finish work at a reasonable time and have some time in the evening. So for me as long as I have the discipline to get my workout in the morning, I feel like I have plenty of time in my life.
Cameron Peron
01:02:36.83
Anything else that you want to share?
Karim Varela
01:02:39.93
Only other thing – to toot our horn here is that we have two high level engineering positions open at Coffee Meets Bagel. One is engineering manager position on our backend, managing five engineers and building out our platform to scale to millions and millions of users. The other one is a senior data scientist helping us optimize our algorithm and take dating to the next level.
Cameron Peron
01:03:11.93
Where can our audience find more about you? twitter, linkdin, fb, instagram?
Karim Varela
01:03:22.37
Linkdin is good. I’m pretty sure I’m the only Karim Varela. I have a personal blog, karimvarela.com. We have a tech blog tech.coffeemeetsbagel.com. Our jobs page is at coffeemeetsbagel.com/jobs.
Cameron Peron
01:03:40.90
Perfect. Thanks.
Karim Varela
01:03:46.83
Thank you, Cameron. It’s been a pleasure.