Developer Experience for Partners & Customers

Between Product and Partnerships is a podcast brought to you by our group the SaaS Ecosystem Alliance, and it’s focused on bringing together product, partnerships and engineering leaders to discuss how to build support and scale SaaS ecosystems. If you're interested in watching or listening in on this conversation, you can access the video here and a link to listen on podcast platforms here.

‍

Our former Director of Marketing, Kelly Sarabyn, interviewed Sid Maestre, the Director of Developer Relations and Advocacy at Lob, about how to think of partner developer experience vs customer developer experience, priorities for organizations first focusing on developer relations, qualitative and quantitative ways for collecting developer feedback, north star metrics & KPIs to track the developer relations function, and arguments for investing in developer relations. 

‍Intro

Sid: I'm Sid Maestre, and I'm currently the director of developer experience and advocacy at Lob. I've been in the developer relations field for about a decade. Previously, I was at Xero, where I did a lot of work with partners and building that ecosystem. I got my initial start with PayPal. So definitely have been around different companies of different sizes.

Kelly: That's one of the things I wanted to ask you about. Developer Relations has been around for a while because there are SaaS products that are for developers. Those types of companies have to think about the developer experience out of the gate, because that's their customer, right? 

In the last, especially five years, even early stage SaaS companies have to start rolling out an API, so that they can be interoperable. Even if their customer is a business user, like a real estate CRM used by real estate agents. Even then, you get to a certain point where that product has to be interoperable with other systems customers are using. 

How should organizations think differently about the partner developer experience vs the customer experience? Can they take any learnings from the field of developer relations that was aimed at customers?

Sid: Yeah, that's an interesting question. To me, a partner is just a special kind of developer. Often when you are building API's for developers you're thinking about authentication, and how to develop an onboarding experience for developer documentation.

You definitely need all of those same things for partners; However, my experience is that when you develop an actual partnership program, the people that you end up talking to aren’t always necessarily the developer. It could be a business development person, a marketing professional who's looking for a channel with which to sell their product, or it could even be a founder if it's a small company. 

These people are really interested in what your partner program is about, the benefits, and what they have to do to become a partner. Especially when it comes to engineering resources, it's an investment. So, they definitely do their due diligence before they start building. They might have some engineers poke around, play around with your APIs, or look at your stack, but they want to understand what the ROI will be if they partner with you. 

That often means they want to talk to a person. I've definitely been on the receiving end of those phone calls. It's a lot of fun, and you get to understand the businesses, the startups, and the cool stuff they're building. I've really enjoyed that aspect of it. 

This, however, is very different from developer evangelism or advocacy where you're out there speaking at events and building code samples and things like that. It's a slightly different audience. It’s like a superset.

Kelly: As companies scale to the size, of say, Salesforce, you'll still have that business partnership person who's coming in first. I generally agree with your point that they want to make sure there's enough of a business justification. Then, the developer comes in to get a sense of how hard is this going to be to build. 

Large companies like Salesforce, for example, have this whole developer community, but it's a mix of customers and partners. It might even be more customers than partners. 

Related Content: Building a Developer Relations Team: Interview with Lorna Mitchell

Moving forward, do you see SaaS companies trying to make the partner experience more developer-first and trying to attract the developer partners specifically?

Sid: I don't know if I've necessarily seen that. I mean, I'm a strong advocate for creating great developer experiences, and I think that should always be a priority. 

I was at Xero for seven years, and I saw a huge improvement in developer experience. When I think back to the first couple years I was there, I was often amazed at what partners were willing to put up with, as far as, things missing and not having everything in place for them. 

Despite this, they were so excited to join Xero’s ecosystem and to connect with our customer base. They saw a real opportunity there, and they were willing to work through things with us. 

So, I would say to smaller SaaS companies that even if they may not have the infrastructure that Salesforce might have, I think they can still succeed if they have a strong value proposition.

Let’s say a SaaS company has at least one person focusing on the developer experience. What are some big bucket priorities that you would advise organizations to focus on first?

Sid: I'll share my thoughts on both the developer and the partner. Across the board, if you ask any developer, they're going to say documentation is their number one priority. When I say good documentation, this can mean how it's presented. Is it written for humans? Is the layout design easy to navigate? Can people get all the information they need? Most importantly, is it up to date and accurate? 

Nothing frustrates a developer more than when they land on some out of date documentation, and they waste a couple of hours only to discover that things have changed. So, documentation is number one. 

Then I’d look at the experience of getting your API keys and that onboarding experience. If you're putting up barriers and roadblocks, like asking for credit cards or saying users have to go through an approval process just to experience your API, I think that is a poor pattern. 

You tend to see that in larger companies like banks, or companies that have information that they are very protective of. I don't think many SaaS companies have that issue. 

On the partner side, I’d say to make sure that you've thought through that partner journey.  You don’t want to stick up a form or email that says “reach out to partner with us,” but then that partner inquiry goes into a black hole or you don't get back to them in a timely manner.

That can sabotage a partnership from the get-go. Think through how you're going to get people in the door, and how to curate a really great experience for them from the partnership and developer side.  

To me, that’s the beginning. There's a whole lot more you can and should do, but Rome wasn't built in a day. We all have to start somewhere, and those are some good places to start. 

Do you have any suggestions for tooling or processes that companies can use to make sure documentation stays up to date?

Sid: You can implement a combination of both tooling and processes. I'm a huge fan of people in the API community who are always beating the drum around best practices. API governance is something we talk a lot about. 

As companies, we don't always prioritize this, but we should have in place policies on how new APIs are build, how old APIs are deprecated, and how these updates are communicated to developers. That's the process side of things that I think is important. 

Then, an area that I'm been pretty passionate about is API specifications. I know that sounds really nerdy, but when we were at Xero that was something that I championed internally and helped build. We basically reverse engineered specifications, and then used those to start building SDKs that were supportive of multiple languages.

Whenever there was a change in the API, or we discovered a bug in our specification, we could update it there, and it would roll out in multiple languages. Then, interestingly, the team that was managing our documentation saw that project and rolled that into their process. 

It's now become a single source of truth around what the description of our API is and when we change our API, we also want to make sure we update our specification. 

I think at Xero, they also built their entire API interactive Explorer from the specification as well. They're really leveraging it in some major ways, and I think it's delivering a much better experience.

Kelly: I'd love to hear more about the SDKs coming from the specification, because I'm doing this large research project and studying 400 SaaS companies at different stages of development. 

I was interested to find that, in terms of external public APIs, the vast majority of the 100 largest SaaS companies (the top 50 public and the top 50 private) had a bunch of SDKs for their APIs; However, when I went down to the series D, which are pretty sizable companies, the majority of them had public APIs, but no SDKs. 

It made me wonder, because clearly there's a recognition that it is a benefit to the developer to have that as an option.

Related Content: Mastering API Documentation: 3 Proven Best Practices for Success

Is it a ton of work to maintain SDKs over time? Or are specifications a way to make that manageable? Is there a tool that facilitates that process between the specification and the SDK? 

Sid:  I'm super interested in seeing the outcome of the research. Add me to the list for that. But yeah, this is a really interesting question. In 2014 I was at Xero, and we had a couple SDKs that were in various states of quality and were owned by the engineers in their free time. 

There were also some community SDKs that were written, but nobody really owned them. I remember talking to the people who ran the platform team, and I brought up that we should be offering some official SDKs for the six major languages out there.  

The answer I got back was super interesting. They said that our API support ends at the endpoint. They didn’t say it in a way that was trying to shirk responsibility. It’s not that they didn’t care, we were just such a lean team. 

They had to focus all of the engineers on building functionality for the APIs. They didn’t have the bandwidth to build all these SDKs and own them. Then, a few years later, Xero is really growing, and they started exposing APIs for banks. 

I ended up on an email thread where a bank was asking us for our Java library, and we just had a couple like random snippets of code that we were sharing with developers. Honestly, I was kind of embarrassed. I thought we had to do better. If banks are adding our API to their infrastructure we should level up our game. 

That led me into this journey of hand crafting the very first iteration. Then discovering Swagger and Open API specs, I thought, this could be a way to really automate those improvements, and get more robust coverage initially in the Java SDK. Then that spread to other languages. 

It was an investment, especially a big time investment. There is an argument around when is the right time to do it. You can also argue that with so many great open source libraries and talented developers, they could just pull down an OAuth2 library. As long as you're following the standard, they could figure it out. 

A lot of startups are very comfortable managing their ecosystem in that way; However as your pool of developers grows, and you get more and more technical questions, you might realize that you can't give any standard support, because there is no standard library that your developers are using. 

They're using one of a dozen Java HTTP clients out there, and then now you need to be an expert in all of those, or hope your community will chip in and support each other. It's hard to have all that technical mastery inside your company. 

My perspective was to limit it to six official libraries, that we can and will support. That enabled us to shrink our support footprint. As your developer community grows, there might be that tipping point where you want to do that.

Kelly: I love your thoughts on developer community in general. For SaaS products that are used by a business user primarily, and are trying to build interoperability for product integrations, it's a lot harder to build up a partner community. 

For example, I’m a Webflow user, which is primarily a no code tool, and they have released some APIs, but it’s not super robust at this point. If you ask a question in their forums, which are quite busy for the normal use cases, there's not a lot out there. 

If you don't have a developer product, where customers can also engage in the developer community with partner developers, it can be hard. When should organizations invest in building a partner developer community?

Sid: That's a really great question. I think what you're describing is that they're missing a critical mass. The question here becomes “does the company have the resources to be there to seed these conversations?” It's hard to get that momentum going unless you invest in it. 

Communities don't just appear out of nowhere. It feels like in your example with Webflow you're a bit of an early adopter of these more developers-centric features, and the community historically has tended to be more designers. 

We use Webflow over at Lob, so we've got someone on staff who's super talented and into that community. Maybe I should introduce you to her, she's super smart. She's always giving us tips and figuring out how to make magic happen with that tool. But, yeah, that is a part of developer relations and part of the Developer Advocate role is to be that initial touch point and backbone for the community. 

That is, until the community has matured to the point that you can take a step back, and observe the growth and organic magic that can happen. Companies will put out SDKs, or open source projects that they use internally on GitHub, but if nobody invests or owns that, it can turn into a ghost town around a project. This happens all the time. 

It’s a bad look when you've got a bunch of unanswered issues, and open pull requests that nobody's responding to for six months or more. It makes the company look bad. I would say that if you're going to build a community, make sure that you have those resources and ownership. 

Be there to support because otherwise you're setting yourself up to look like you don't really care about that segment of your audience.

Kelly: Yeah, that makes sense. So, I think we all know how common REST API's have become, but I was surprised when doing my research that it's even more common than I would have predicted. The larger companies tend to have REST, but also three or four other styles; However, if you take your typical SaaS company that started in the last five years, a lot of them are using REST APIs. 

Part of this is probably because more developers know how to work with that standard. Especially when you have partners who might just be building one integration, instead of using your product every day, it's an advantage to have a standard that more people are familiar with. 

Have seen any other emerging standard, styles, or protocols that you think are the next big thing? Something we might not be aware of that people are already adopting in the space of making products interoperable?

Sid: Yeah, I think that you're touching on something that is really true, which is that RESTful APIs have become the standard. It's always good to look for that next thing, but I feel like REST is the backbone of so many SaaS applications. 

What I hope is happening today, that didn't happen 5 or 10 years ago, is that teams building products, and that front end that's talking to a back end through APIs, are thinking about how to leverage the same interface for developers and third parties. 

What typically will happen is they'll build a tightly coupled application, and not really think about publicly exposing it. Then suddenly, in a strategy meeting, someone will say they need to open up the platform or they need to become a platform. Now they’re not just a product, and they need to open up all these API's. 

Then developers are like “Oh, we didn't build our API's with that in mind.” Now they are forced to either refactor in a major way while customers continue to use these APIs or they do the opposite, which is rebuild them for public consumption. 

Then there’s a real challenge of continuing to open up new features inside the product, and also selecting what to open up to your developer community. For me, REST is always going to be there, but if you can build API-first, and think ahead to that possibility, you're setting yourself up for success. 

As far as new technology, I know there was so much buzz about GraphQL. I think it's come back down to reality where GraphQL is really great for specific uses. If you're trying to pull multiple pieces of data from different places, and you don't want the developer to have to make a series of a dozen API calls to collect their data, you can construct those those queries. GraphQL is a great solution for that. 

I wouldn't say get rid of REST and replace it with GraphQL. No one in their right mind would say that, but I think it has its place. 

What I think is much more interesting is webhooks, and more event driven architectures, and the different protocols that you might want to explore and make available for your developer community. Nobody wants to pull a RESTful API every couple minutes to see if there's any new data. It's much more efficient on everyone's behalf to get those events sent to you. 

A lot of companies tack on webhooks at the end, after they've been around for a while. I’d suggest that if there's a real strong use case, and you could start offering those sooner, developers would really benefit from having that notification, and being able to subscribe to the information that they're most interested in, instead of having to pull a RESTful API.

Kelly: Yeah, I'm definitely seeing a lot more companies offer webhooks. Even if they are just offering it for four or five outbound events, it's at least something. Presumably, they're choosing those use cases that they're getting the most feedback on needing. 

I'd love to hear your take on collecting developer feedback. When it comes to the partner relationship, you have business stakeholders who may have certain needs saying one thing, and then you may have the developers who may be behind the scenes, and not in close contact with you, who might be having a different experience. 

They might think your rate limits are horrible, and you’re sabotaging their use case. 

Related Content: What are Webhooks?: An Explanation for the Non-Technical

Once you have partner developers that are using the APIs and building integrations, what's the best way to make sure you're collecting their feedback in a systemic way and making sure it's incorporated internally into your organization? How can you make sure you’re not just getting the noisiest people, but more holistic feedback? 

Sid: You're, touching on a sore topic for a lot of developers when you start talking about rate limits. If you do start building libraries and SDKs, I’d suggest having some way to help with managing rate limits, reading those header responses, building in retry and timeouts to wait to then retry after. That can all be addressed technology wise.

When you talk about feedback, there's so many different ways that you can approach that. With online communication, and the ways that I've received feedback from developers, it's true that it’s sometimes the loudest, most angry developers that you notice. 

Interestingly, if you get into a conversation with them, usually within a minute or two it becomes the nicest, most productive conversation. This is because they are so happy that you're listening to them. Once they get the steam out, they then give you amazing insights into how they're experiencing your product, what the real challenges are, and specifically what they would love to see changed, fixed, and improved. 

I've done that before where someone was really angry on Twitter and I reached to them via DM to jump on a quick 15 minute call to talk through their problem. That's more of the one-on-one engagement. Forums are another idea. Have some kind of developer forum where people can post questions. 

Through those forum questions you can start to discover common problems or common threads through the community, because people will jump onto that thread and they'll share their frustration or their stories. 

You can even use tools out there to solicit feedback, and let people vote-up on different things that are important to them that they would love to see fixed. That's all online, but what I found is talking in-person makes a difference, and I’m hoping we can all start meeting more in person again. 

In the past, we've done road shows for developers where we traveled to different cities. Xero used to do these big Xero Con events where our partners would come and they'd exhibit. I would spend the entire event walking around, talking to different partners and building a personal connection. I’d also get to hear what they loved about the program and the the APIs, and about what their real pain points were. 

We also brought in a select group of partners to do a 20 person roundtable. What was most interesting was the way that all they fed off of each other, and built off each other's ideas. So, if you get a group of people together who all have the shared goal of being successful in your ecosystem, they can start to build on each other's ideas, and start coming up with solutions and great suggestions for you. 

I found that extremely rewarding. There's so many ways you can slice this, but it depends on the size of your community, and the resources you have available to go and engage with them. No matter what you do, try to engage with them and be open. Listen, ask questions, and be curious. I think it's a great way for you to improve your product and your API's.

Are there ways that you can collect more quantitative data, like API usage, to gauge where there might be problems? 

Sid: Yeah, I absolutely think that giving folks who are building the APIs access to the raw API logs can help with figuring out ways to analyze and look at different patterns. At Xero we would often look at 400 and 500 error codes. 

Often those would surface on an individual level. You could definitely do it across your entire developer base. Some of the things that we discovered was bulk creation, and  that was something that we enabled. 

We also had a problem where we only gave back a slimmed down set of data when you called to get a list of all your contacts, for example. If you wanted like to dig down and get all the address data, you'd have to call the endpoint for each contact one after the other. 

That's where those rate limits came in. That’s a lot of API calls if you have thousands of contacts or invoices. We ended up implementing pagination. You could then get back a hundred detailed records at a time. So ten API calls got you a thousand records. 

That was a really big thing, but we had to constantly educate developers because they would just go in and start using the APIs, and they wouldn't necessarily catch that “magic trick” that they could employ to avoid hitting those rate limits. 

Also, I have definitely sat down on calls with partners where they would say they’re hitting rate limits. Then, as we’re looking through their log files, and talking through their code, they realize they’re looping over something, and calling it 100 times when they only needed to call it one time.

And so, just by looking at the logs, and talking through it with them, they discovered that they had a little bug in their code. If they just moved this one API call outside of this loop it totally solved their problem. You can find solutions on the macro, as well as, the the micro when you look at api logs. There's magic in there.

For partnership teams that are trying to scale their program and roll out a developer relations function at their company, what are some Northstar metrics and KPIs that they should put upon the developer relations function?

Sid: That's an interesting one. When you talk about partnerships, it really depends on what your program involves. For example, are you focused on the amount of new partners that are signing up? Are you actually qualifying how many you launch in your marketplace or directory? 

Those were all things that we tracked at Xero. We had been doing that for several years. What we noticed was that partners would get stuck midway through the process, and they would sit there for months and months. We were wondering why they got stuck at that stage. 

We then started digging into our Salesforce data, and started to see that certain steps were where people got stuck. We asked ourselves questions like: Is there a problem with our process? Did we set the hurdle that's too high for people? Is there something in our process that's broken? 

By looking at how quickly they moved from stage to stage, and where people were getting stuck in that journey, this helped us reevaluate and optimize to move people through. So, our metric became focused on how quickly we can move people from stage to stage. 

We started looking at the average time people spent on different stages, and we were able to optimize and have big gains in the velocity that we were able to get partners through. When you're just starting out, stick with some of those basic numbers might be where you begin, because  you're not worried about velocity when you're just starting out a program.

Kelly: Yeah, you might even be more concerned about successfully onboarding partners, and trying to recruit developers to actually build on your product and get that front end number up.Then as you scale you might be more focused on pushing them all the way through and more prioritization and quality control. 

Last question, if people are in a developer relations function, but they don't feel like executives are fully supportive of the business impact, what are some arguments that they can use to demonstrate the importance of this function?

Sid: If this was a program that was already built-up but not getting buy-in, it would be interesting to ask the executives why the company started the program and what did we hope to get out of it. Why was the API built in the first place? As a business, what were our goals and what were we trying to accomplish? 

Get them thinking about why we decided to go in this direction. Inquire on if the strategy changed, and if this function is not a priority anymore. 

If you are coming from a position of trying to defend and justify this, then it's really easy for an executive to dismiss your arguments. You can instead prompt them to think deeply about how this fits into the business. That might be a bit more persuasive.

For me, we always struggled to have that direct line to revenue. When you've got a SaaS product, you can get blinders on as leaders, and you can think it's all about subscription growth and maximizing revenue through that mechanism. 

Leaders might forget that your product is actually part of a world of products, and no customer wants their data sitting in a silo where they’re not able to share it with other apps that they use on a daily basis. Small business customers use on average us 18 different apps.

So, not being able to use Zapier or some other tool to move data from place to place is a real strike against your product. Also, when we talked about it at Xero, the customers who actually connected two or more apps to Xero saw this huge drop off in churn. 

If you want to talk real money, talk about lifetime value of the customer. All in all, I really would hope that executives would be aware of that. If they aren't, you could always prompt them with some of what I mentioned. If they don't remember, you can always try to gently remind them of the power of a connected application.

Kelly: That's great. To your point, if the function was started, somebody believed in it in the beginning. Sometimes, however, people can be short sighted, and leaders can change. I think those later arguments are good for leadership because they connect back to revenue. 

I definitely believe people are becoming more and more aware of those connections as well, and the value of community in SaaS, so that ties into developer relations as well, where a lot of their function is about building that community.

Sid: Yea, and I always go back to the customer. Having an API and ecosystem ultimately delivers more value and a better customer experience. I really believe that it does, and it's hard to deny that this makes your customers happier. Those customers are going to become your advocates. It's hard to measure, but it's it's extremely valuable.

Kelly: I agree. Thank you for joining us today. Is there somewhere that people can find you if they want to read or learn more?

Sid: I'm on Twitter and you can look me up there. You can also find me on LinkedIn as Sydney Maestre. I've also got some YouTube talks out. My favorite one is “Structuring Partner Programs for Success.” I did that talk at Nordic API's in 2018. 

End

If you enjoyed this interview, check out our Youtube channel and subscribe for more content on all things APIs, integrations, and technology partnerships. If you're someone who is working on building and scaling SaaS product partnerships, we invite you to apply to be a member of our community and network with other leaders like Sid working on this at SaaSecosystemalliance.com.