Our new Grades feature gives you perspective on the performance of your endpoints, relative to other Skylight apps, so you can focus on meeting your customers' expectations.

Working on performance is always a tradeoff. The truth is, you can always invest more resources in improving your app's performance. While you can indeed lose customers for offering a subpar experience, most of them are probably not here for that experience alone. At some point you reach diminishing returns and you are better off spending your time on adding features for your customers.

Therefore, to really make sense of your performance data, you have to put the numbers into perspective. It's good to know that your endpoint's typical performance is 250ms, which is certainly not instant, but is it fast enough? Would it meet my customers' expectations? How many extra miles would I need to go to really blow them away?

The new Grades feature in Skylight is designed to help you answer those questions. To help you understand how your endpoints are doing relative to your peers, we analyzed the performance data across the Skylight customer base. With that information, we were able to assign a letter grade to your endpoints based on their place in that distribution.

You can preview this beta feature today by enabling the Grades feature flag.

The Future™

We are quite happy with our first iteration on this feature, but we have plans to do a whole lot more.

While it is useful to know how your endpoints are performing against the general population, sometimes it could produce unrealistic targets. Sometimes, there might be intrinsic or practical limits on how efficient you can be. In these cases, your customers might already be accustomed things taking a certain amount of time.

The holy grail would be for us to identify the factors that lead to those differences and perform the grading within those groups. While that level of fidelity might be impossible, there are plenty of things we can do that gets us closer to that ideal.

There are two avenues that we are exploring. First, we need to collect more data about your endpoints. In fact, we have already started doing that with the introduction of the segments feature in agent 1.2, and there will be more to come. The second avenue is to incorporate more Data Science™ into how we analyze and aggregate your data. I will save the details for another blog post, but there are a lot of interesting things in this area!

The Grades feature marks the beginning of a direction we are quite excited about. At Skylight, we take pride in building a performance product that provides Answers, Not Data. We will keep working hard to live up to that motto!

P.S. we are going to be at RailsConf this week. Yehuda and I will be announcing some very exciting updates to the Helix project, while Vaidehi will be telling the story of Goldilocks And The Three Code Reviews. We will also be demoing some cool new features at our booth. See you there!