Metrics & Dashboards

Tracking and measuring your company’s performance is crucial not just at the early stages but through the course of your companies lifecycle. in this episode of Matrix Moments, Tarun Davda & Vikram Vaidyanathan, Managing Directors at Matrix Partners india, share learnings from their experiences with founders on why tracking the right metrics can help you measure what is working for your company and what isn’t. in addition to measuring performance, picking a North Star metric as the one metric to align the entire company on what is important helps keep all teams focused on working towards the same goal. Listen in to know more.

Rajinder: Hi, and welcome to Matrix Moments. My name is Rajinder and i am a director at Matrix india. And i am joined today by Tarun Davda and Vikram Vaidyanathan, Managing Directors at Matrix, india. We are going to discuss a topic that early stage founders spend a lot of time thinking about: metrics and dashboards. Guys, why do VCs love measuring everything? i mean absolutely everything?

Vikram: So there are many types of VCs. VCs come in all shapes and sizes. So, there are data junky VCs who measure everything. There are FOMO VCs who measure what other people want. There are vanity VCs who are only measuring vanity metric. And then, there are financial VCs who only measure financial metrics. And i think i have been accused of being all of them. But what’s common to everyone is that they like to measure and measure something. And i am going to toss this to Tarun. And let’s talk about why founders should measure. We know VCs like to measure everything. But, why do we think founders should measure how their company is doing?

Tarun: Hey, guys. This is the Tarun. So i think what Vikram said is true. Every VC likes to measure. i actually think that every founder likes to measure. if not, they should like to measure because i fundamentally believe that you can’t improve what you don’t measure. A lot of what you do as an early stage founder is gut based. And a part of that is true. But, the reality is that while things are going well, it’s great to be gut based and basically use intuitive understanding of your customer to improve the product. But, things don’t always go well. And we know that that’s true in more Startups that there are more challenges than things that are going well. And so if you aren’t measuring, you essentially don’t know what is going wrong, why it’s going wrong, and where the engineering and the product team needs to put more focus. So, that’s number one.

The second is as a management team you are always looking to align the entire company on what is important. What are the levers that the team should be focused on in terms of making either getting the product market fit or making the business model work. And so, unless and until you don’t have a metric that you can align the entire org on, it’s very very hard to sort of drive towards a particular outcome. So that’s i would say point number 3.

Next is if i look at overall every company, depending on the stage of the company, either you are iterating towards PMF or you are trying to scale PMF or you are trying to become profitable or you are trying to sort of get both because you have already figured out profitability. You figured out the business model. And you are now trying to scale up. At different points, having some indication about should i be pressing the brake or should i be pressing the accelerator, is very very important. And most management teams are looking for data to tell them that. And we have seen in some very early stage companies when they don’t measure this, they actually don’t know, for example if you are burning more money with every transaction as you are growing, ideally you should be pressing the brakes and trying to figure out - get that piece before you start accelerating again. if you don’t measure, you won’t know that. And so sometimes you could actually take wrong business decisions.

Last, at every stage as an early stage company, you are going to start projecting few quarters out what the business looks like. if you haven’t been measuring sort of the building blocks of the business on the product side, on the marketing side, and growth side, you will not be able to have enough understanding of what assumptions are to make while building the business plan. So if you don’t know that conversion rate today is X%, how are you going to be able to project it out 12 months and say, hey, you know what? Here is where i am. Here is where the industry benchmark is and hence here is where i need to go to.

Now if you are measuring it, you will know how far you are from the benchmark and what you need to do about it. But more importantly, it gives you a view into the feature and say, hey, 12 months out, here is our aspiration of where we need to get to. So actually, the dashboard also over time becomes a building block for your business plan in the future.

Vikram: i will just add to that business plan and often people ask how do i make sure that the forward looking dashboard is just not aspiration. And to me, the reason to make a plan is to know when that plan has changed because - either you are going to be way ahead of plan or way behind plan. You need to know which is which, in order to press that accelerator or the brake like Tarun said.

i am going to add a few things on sort of the why measure and also how. The first thing is on instrumentation. And, often people are scrambling to put numbers into a spreadsheet when the MiS or a dashboard is due. What’s better is to say i am going to build this into the tech. For example, let’s say you want to measure turnaround time of a loan. it’s easier to actually put that in the instrumentation of the loan management process, so the TAT becomes a metric that is measured as loans are getting processed. That’s what i mean by the instrumentation. And instrumentation will be different for a Fintech company versus a logistics company versus a marketplace. But getting that instrumentation right once you have decided what the dashboard is actually critical. Otherwise, you are just going to be scrambling to fill in a spreadsheet.

The second is including the product tech teams in the design of these dashboards. And often the dashboard is a business team’s KPi or business team’s deliverable. And the product tech team is doing their own thing. if you include the product tech team in the dashboard, one, you will get the instrumentation right. And two, you will also carry the product tech team along on what the business KPis are. And so for example, one of our logistics companies included the product tech team in designing the logistics dashboard. And they started looking at utilization as a metric very differently. And they went back and started redesigning the route optimization algorithm very differently once they knew, oh, you know what, this is the business output or the performance output that we are looking for.

i think it’s even more important in what is called phygital businesses which is your physical plus digital where you have a tech team and a financial services team and they are not really understanding how the other guy is thinking. So making sure the product tech team is actually part of designing the business dashboard becomes even more important.

Tarun: i will just add one point over there, Rajinder. i think the mistake i have seen a lot of young founders make is they think that dashboard is meant to be built when you are ready to fundraise for the investor. And i think that is a mistake, most quality investors when they invest in your company will guide you against it saying that guys the dashboard is actually for you as a founder as a CEO to manage the health of the business on an ongoing business so that when you are ready to have Fundraising conversations, it’s very clear that you have a grip on what the company is doing and what each metric is doing. So i think i also want to dispel this myth that a dashboard is for investor reporting. it’s not for investor reporting. investor reporting is a byproduct. The real aim is to help manage the company on a daily, weekly, monthly basis.

Rajinder: So actually that’s a good point. So i was going to chat about cadence. Would love to get your views on what’s the right cadence for companies to actually measure their business health. Often founders are trying to figure out the changes they are trying to make and when to evaluate whether those changes have actually made an impact? How do you advice your founders on cadence for metrics?

Vikram: So it’s a great question. i think that cadence can actually drive execution as well as iterations for companies. Some metrics will honestly be weekly metrics. For example, DAU in a social company or a vernacular social company will be actually a weekly metric. Some metrics are clearly more call it monthly or quarterly metrics like NPS. You wouldn’t actually measure NPS on a daily basis.

So i think one is to define some of these metrics. And some of these which are driving sort of daily or weekly product iterations will be in the daily/weekly bucket. Others which are going to decide monthly/quarterly priorities like NPS which should be in the monthly/quarterly bucket. Tarun, i don’t know if you feel differently on it.

Tarun: No, i think it comes down to a few things. Essentially, the way i think about metrics is there are growth metrics. There are profitability metrics. There are usage metrics. And essentially, some version of your MiS is essentially targeted towards each of these three large buckets. Now there be certain like growth metrics that you will want to measure on a weekly or a monthly basis especially in very early stage companies. As your company becomes more mature, it will turn into monthly either metric or a quarterly metric. But when we were in China recently, one of the founders told us that the thing that’s unique about China is they actually measure week-on-week growth, whereas a lot of founders in india focus more on monthly growth.

And i think it is also about the nature of the market you are in, the stage of the company and the kind of metric. To Vikram’s point, something like a NPS which is more about usage and stickiness of your product, it’s unlikely that you will want to measure that daily because that’s not something that A) should change daily unless you are really doing something drastically different. But, it’s more about a health metric that you will measure more on a monthly or a quarterly basis. But, something like, for example, conversion rates on your website or on our mobile app, for example conversion rates on number of loans that you are processing compared to the incoming applications. These are things that you will want to have a tighter control on because if something has changed either in your product or in your tech at the backend, you will want to very quickly be able to react to that so that there is no sort of adverse business impact.

A lot of this context specific where certain things just make more sense to measure on an ongoing basis. But, certain things you will want to measure more over a period of time.

Rajinder: Let’s shift gears to the second question which is what to measure? Many companies struggle to find the right balance between tracking too many things versus tracking just a few key things. And talk about how many metrics company should be tracking also in the context of what stage of evolution they are, early stage companies which are looking to find product market fit versus let’s call it growth stage companies which have some version of product market fit but are really trying to figure out the business model. How deep should this metric tracking really be?

Tarun: So let me take that and i would love Vikram’s views on it as well. So for very very early stage companies which are typically like at seed stage, which is generally where either we would enter or a series A where we would enter. So mostly for seed stage companies my advice to founders is the most important thing is to understand your North Star Metric, and North Star Metric is not generally an operating metric. it is something which is much more fundamental that is centered around the value that you are able to deliver to customers. So for example, one of our portfolios companies Belong very early when we invested the only North Star Metric for them was the number of job offers that were made through the platform.

Now they weren’t even monetizing the number of job offers and the business model wasn’t centered around each job offer resulting in some sort of revenue for the company. it was much more of a subscription business. But the reason why the number of job offers was important was because it actually showed there is enough value that the platform is delivering. And if there is enough value the platform is delivering, the best metric to reflect that was are people actually able to find jobs and make offers through this platform. And hence, the number one metric for a long period time just stayed as number of job offers being made through the platform every month. it wasn’t timed to any sort of operational data. it wasn’t timed to any sort of financial outcome for the company. But that was the number one metric that allowed us to know are we moving towards PMF. So that’s number one.

i would say more generally for typical early stage companies, my recommendation is try and limit the things that you measure to five, maybe five key metrics. Under no circumstances more than eight metrics because if you measure too much, you essentially don’t know which one to optimize. And so focusing on those five, six, seven key metrics, maybe one or two related around growth, maybe one or two related around sort of unit economics or health of the business, maybe one or two around sort of retention or stickiness or sort of usage of your product. But anything more than this is too much. Especially again like i said this is largely for early stage companies because if you are measuring too much essentially you are getting way too much signal and there is an analysis paralysis kind of tendency that you may fall under which i think is dangerous for early stage companies.

So i would do just that - figure out what are these five to eight key metric for your company. Segment them under like i said growth, profitability, and usage and then figure out which one makes sense.

The last one i would add is basically it’s different for different businesses. So a SaaS company should be measuring different things than a media company. A media company should be measuring different things than an ecommerce company. You will mostly find that every industry there is enough knowledge now or if you talk to few founders who have got some more experience in your domain, which metrics to measure is fairly clear. Equally important is figuring out the benchmark for that metric. So for example if you are looking at retention in a social app, what is the benchmark for retention for Day 30, Day 90? Having an understanding of that is equally critical for a young founder because the metric in isolation of knowing what’s the industry benchmark for best-in-class is meaningless like you could be at 15% Day 30 retention, but you don’t know whether that’s good, bad, or ugly. And so knowing the benchmark for that and knowing what is the top decile companies in that vertical doing, i think is equally important.

Vikram: i think on the benchmark’s point, what i would add is a concept of market awareness. Be aware of how the overall market would sort of measure you. And often, we are in early stage technology companies which are disrupting incumbents. And we are usually measuring something which is different from incumbents. But it’s always important for us to be able to translate it into a different benchmark. So for example, again in Fintech , financial services companies are usually balance sheet business while Fintech s are not balance sheet businesses. But it’s important for us know how you will translate your current metric maybe throughput and flow of loans and flow of applications into a more traditional context. And therefore then say that i am going to measure because this actually translates into the traditional world into a different metric. i love this five to eight metrics that Tarun just said. The only thing i would add is i always like to have a good mix of input or health metrics versus output or performance metrics.

And the one advice i would have for early stage founders and early in their company’s evolution is to pick more health metrics. So for example, a good health metric input in SaaS company or a even a Fintech B2B company is salesforce productivity which actually gives you a good sense of whether your salesforce is working efficiently or not. And then later on, work on more output and performance metrics whether it’s disbursements or AUM and so on that might come a little bit later in your company’s evolution.

Rajinder: You guys are making it sound really easy. And i know it’s really hard from many conversations that founders have shared with us. Talk more about some of the non-intuitive things you all have learnt about this North Star metric you brought up. What are some examples of things that founders have spoken to you about which are quite different from what you would have otherwise imagined about the business?

Tarun: So you know let me take maybe three examples from some of portfolio companies. So i think sometimes the North Star Metric is a straightforward like the one i shared with you which was the wrong example which was basically number of job offers through the platform. Sometimes the North Start Metric would also be a derived metric which actually leads to something which is much more fundamental in terms of your business model working or not working.

So i will tell you in Ola, obviously early days the thing that mattered the most was the number of rides that people were taking. The other thing that mattered the most was are drivers sort of sticking to the platform, and are drivers loyal to the platform. But, whether drivers were loyal or not was not an actionable metric for us. it was like, okay, drivers are retaining an X percent. The question was why are they either staying on the platform or why are they not staying. And what is the difference between the two. it was a little hard for us to initially sort of get our hands around.

And then we came up with this metric which i think was very helpful in the early days, which was driver earnings per hour. And it fundamentally came down to if the drivers were making more than X rupees per hour, they would stick. if they were making less than X rupees per hour, they would not stick. And that fundamental insight and the moment you are able to get to that answer and say, hey, you know what? By city, by category, by sort of micro-market, if i am able to track that or even sometimes at driver level, if i am able to track that which driver is actually making more than my benchmark earnings per hour and which drivers are not and what do i need to do. To me, that’s a very actionable metric. Whereas, driver retention is a percentage and there is not much i can do about it other than knowing that, oh, it’s worse than last month or better than last month. it’s not actionable in anyway.

And so for me, i think just along with other metrics we used to measure is driver earnings per hour. And then that sort of broken down by like i said city and category et cetera was something which was very helpful.

The second one was in the case of if i take some of our media sort of social companies, a lot of people focus on DAU, and i think that’s important. it’s very important in the early days because essentially it does show sort of pace of growth. But to me, DAU in the absence of knowing what is your time spent or MOTS (minutes of time spent) and retention actually can be very misleading because you could be adding a lot of DAU either organically or inorganically. But if the DAU is not sticking to the platform or is spending very little time on the platform, to me those are much more instructive as a metric of the health of the business than just DAU itself.

And so for me, for example, in some of your companies, the focus very early on is how time are spending and how much time they spending 4, 5, 6 hours totally on social media TV, what proportion of the time is being devoted to your platform becomes very important because it shows in some ways it is an indication of the value that your current platform is providing them.

The second is retention, otherwise you are in this treadmill business where you keep adding DAU, you keep running your marketing campaigns. A bunch of users come but essentially after 30, 60, 90 days if all of them are disappearing or a large proportion of them are disappearing, essentially at some stage they are going to hit a growth ceiling where you will just not be able to grow unless you burn a lot of money. So, that’s the second one.

The third one is i would take the example of budget hotels. A lot of us focus on either what’s your average room rate. A lot of people focus on sort of utilization or occupancy. And to me actually the merger of those two metrics is actually the one which is very relevant because you can drop prices and suddenly get high occupancy levels. Or, you can keep your occupancy down, but charge a lot per room. To me, again in isolation the two metrics mean nothing. So we actually came up with this metric called RevPAR which revenue per available room night. Which is at a particular price point, for a particular occupancy level, what is your RevPAR? And to me that again is an important metric because month on month, i can keep increasing prices or reducing this thing, but obviously there is a tradeoff you are making with occupancy.

And so, this actually adjust for that and says if you are selling a room night at Rs. 2000 and it’s at 70% occupancy across your portfolio, that’s actually a Rs. 1400 RevPAR. And to me, that number becomes more important than looking at either of the other two metrics in isolation.

Vikram: i think he has given you great examples on some of these North Star Metrics. i also want to differentiate and add on what i call ‘press the brake metrics’ and ‘press the accelerator metrics’. So press the brake is that you define certain metrics and you say if this metric goes below certain level, i am going to stop growing. So, NPS and churn are great examples of that. And at some point of time, Flipkart famously said that if our NPS goes below this, we will stop growing. And great size company’s say if our churn goes above this number, then i am going to stop growing.

And the other is the press the accelerator metrics. And one of the metrics that Tarun said which is if you have engagement and minutes of time spent which are growing as you are growing, that’s a clear indication that you should be pressing the accelerator and growing much faster.

Rajinder: Final question on this podcast. i think many founders are going to find this North Star Metric idea really interesting. What advice do you have for them on how to come up with this metric? And once they have come up with it, how to translate it into long-term impact?

Vikram: it’s really great and tough question, so i am going to take a crack at it. Often this North Star Metric sort of emerges - and i am trying to go back to the time when how did these things emerge. i think the answer is in the unit economics of your business model. Often Startups are investing far ahead on fix cost and marketing before the end state business model emerges. But there is always a unit measurement and what you are trying to solve for is what is the unit economics of that which will lead to the end state business model. For example, minutes of time spent, if you get more time then you get more money. if you can do more deliveries in per hour for a logistics company or per delivery boy per hour, then you crack that unit metric of your business.

So i think, think very hard for what is that unit of your business which you are trying to scale and how do you measure that unit of business. i think that might be the secret to getting to this North Star.

Tarun: i will just add one point. i think great points and the only addition i would say is that what i have seen in early days in a company, start-ups are chaos. There is generally way too much ambiguity. Everybody is trying to figure out sort of in their respective function what is it that they need to work on. Operations is focused on something. Product is focused on something else. Engineering is trying to build the platform and build some scalability or some stability. What generally founders try to do or some of the best founders i have seen is actually come up with this one North Star Metric which they use really well to rally the entire org around a common goal.

And so, whether it may be MOTS in a media kind of company or it may be driver earnings in a company which is trying to sort of focus on retention for that particular quarter or that particular month, it actually becomes a way for the founder to get people closer, much more aligned. Everybody is clear on what they need to achieve. Everybody understands the impact of that metric on respective functions. Functional leads understand how they to translate that down to their next line.

And so i have actually seen that this North Star Metric concept is a great way for every single person in the org to get clarity in terms of what is it that is going to make this company successful. it’s not just a metric that the founder of the CEO, or the senior team is basically looking at and is basically sharing with their investors . This North Star Metric actually has a potential to get every single person in the team. Everyone from the CEO down to customer service rep. Every single person in the team understands what is it that we need to do.

Vikram: Final thing i would add is to align the investors and board on what you are measuring especially your North Star Metrics. And sometimes every investor has a pet peeve and sometimes you have to add that one metric. But i would say align everyone and make it part of your vocabulary both internally and externally for it to really become your North Star.

Rajinder: Thanks a lot Tarun and Vikram, and thank you all for listening.

Salonie: Thank you for listening. And you can find the transcribed version of this podcast on matrixpartners.in. You can also follow us on Twitter and Linkedin for more updates.