In October, I gave a talk at API World called "10 Mistakes to Avoid When Building an API," which was a compilation of mistakes and missteps the HelloSign engineering team made while building our API. It was a hit!
But wait, why would we want to share mistakes?
Our engineering team has embraced the company values of “Transparency” and “Constant Improvement.” We regularly hold post-mortems to brainstorm ways to fix, improve, learn, and grow. This talk was actually a result of many post-mortems that were boiled down to a handy list of not-so-obvious lessons that others might be able to benefit from.
Below are the mistakes and lessons covered in the talk at API World.
1. Mistake: We built new API features before writing the documentation.
This was a mistake because over time, new features were proposed and built without considering how they fit in with others. Our API grew some gangly inconsistencies that were expensive to iron out after they went live and users started building integrations.
Lesson: Write documentation first.
Feature specifications and design docs don't always address the external developer experience -- things like: Is the URL structure consistent? Are the methods consistent? Is the naming convention for parameters being followed? We write API documentation first when building new features and we encourage you to do this too. It gives you a user-focused platform from which you and your team can decide what exactly to build.
2. Mistake: We didn't notify users of API changes.
In 2014, we rebranded HelloSign and changed the appearance of our site including our logo, font, and colors. This also changed the way our embedded API components looked – flows like embedded signing, which were embedded in our API users' sites.
You may be thinking that a redesign isn’t exactly a big change. We assumed this too. Some of our users disagreed, though, and had either designed their sites to get consistency with our embedded signing flow, or they were just caught off guard and concerned that we hadn’t notified them of the change in advance. Customers wanted to know why we didn’t give them a way to test or opt in to these changes before they went live.
Lesson: Notify users of changes and provide opt-ins.
We learned from our experience and when we rebuilt our signer flow to be optimized for mobile devices last year, we versioned it as a “v2” and made it completely opt-in. Existing users had to elect to transition to the redesigned flow with a version parameter they'd pass with their API requests.
3. Mistake: Waiting too long to provide language-specific SDKs.
We built our API in 2011 and went into beta in 2012, but didn’t release our first SDK until 2014. As soon as we did, our API adoption rate jumped. Obviously SDK’s save development time, but they also take care of the data modeling and parameter management, in addition to making the HTTP request. It’s certainly possible to write this kind of stuff from scratch, but it means absorbing the API docs and that’s time dev teams don’t have or may not want to spend. To many users, your SDKs are your API.
Lesson: Build SDKs as early as possible.
If you don't build them, the community will build them for you, leading to a scattered developer experience that is out of your control. As of today, have six official SDKs, which you can use to easily integrate eSigning workflows into your app.
4. Mistake: Ignoring task automation services.
Services like Zapier, CloudElements, and IFTTT make it easy to connect APIs together. Over the years we have been approached by task automation services looking to partner and we have repeatedly chosen not to because of the bandwidth of our small team and the fact that we saw there being a clear delineation between our API and these services, which were essentially just 3rd party integrations. Today, they provide nearly 10% of our API traffic.
Lesson: Partnering with task automation services will/can broaden the integration options for your users.
5. Mistake: Not dog-fooding our API.
We explicitly chose to build our API separate from our main web application, so this wasn't necessarily a mistake as much as it was recognizing that the amount of time required for QA, Product, and Developers was practically doubled as we had to apply it to both our products and our public API. Dog-fooding (or "drinking your own champagne," as our marketing team likes to say) can save you time in these areas as you only have to do it one time.
Lesson: Drink your own champagne.
Build your application on top of your public API when possible. We’re moving HelloSign in this direction and all 3rd party integrations we’ve built are already based on the API and we've seen significant time savings there.
6. Mistake: We gave everyone the same rate limit.
Our basic rate limit of 500 requests/hour was sufficient until we started growing and onboarding larger customers. They quickly ran through these and would reach out to us for an increase. Unfortunately, when a user needs a rate limit increase, it’s because they just hit it and their application is blocked or broken until they can make API requests, and we did not initially build a quick way to update this.
Lesson: Build flexible rate limits.
Also differentiate between your API requests and consider placing your more expensive calls (e.g., database writes or file processing) under a separate rate limit from your cheaper calls (e.g., reads).
7. Mistake: We incorrectly ignored requests as duplicates.
We don't service requests we think are duplicates, which helps us avoid race conditions and unnecessary overhead. But this kind of bit us because we were too naive in our detection of duplicates and ended up rejecting legitimate requests.
Lesson: Let users define uniqueness.
One way to do this is through an “idempotency key,” which is passed in by the user. The first time a request is serviced, the results are stored in cache using the key. Future requests with the same key can simply be served from cache.
8. Mistake: Not having a plan for API Support.
API-specific questions that came in were initially answered by our engineering team. This actually wasn’t the mistake, we think this absolutely the right way to handle API support in the early stages. However, without a plan for transitioning that support off our dev team as our user base grew, we ended up having to scramble to address the fact that our dev capacity was getting eaten up and developers weren’t happy with the amount of context switching they had to do.
Lesson: Plan for supporting your API customers early on.
Start this search early because it’s not always easy to find people who have that unique combination of technical fluency and empathy and customer voice required for interacting with your API users. We ended up finding the perfect fit for our first API Support hire in one of our existing support reps. Not only did he have a great grasp on the existing product, he had the technical prowess and empathy needed to support our API customers.
9. Mistake: Not providing insight into data and usage.
Users would submit support tickets to find out things like why a request failed, or how many API requests they made last month, or if we could replay certain webhook events. Because our engineering team was servicing these requests in the early days, we paid for the lack of information in dev hours.
Lesson: Expose data and usage to users.
We've done this as part of our API dashboard, which allows users to view requests and responses and troubleshoot errors.
10. Mistake: Only accepting parameters one way.
We took a strict approach to the way we accepted POST parameters, and limited the content type to multipart/form-data. This is the format that’s used when you upload data and files from an HTML form. Our application framework makes it easy to process this kind of request. We also require document data to be submitted this way, so when you're populating a PDF form, for example, each field is its own POST parameter.
One of our customers, Binti, helps parents prepare documents to adopt a child. When submitting all the document data to our API, it ended up being between 2000-3000 POST fields. Suddenly, we were either timing out because we were processing so much data, or the request was flat-out rejected by our web server for exceeding our maximum number of POST parameters.
Lesson: Consider being more flexible in how you accept parameters and data.
One example of how we've put this into action is giving users the option of uploading a file to us directly or just giving us the URL of the file on the Internet, which we’ll then retrieve.
There you have it!
It’s a ton of learnings smushed together into a list of ten. In reality, though, a lot of the learnings have similarities. If you take nothing else away from this post remember:
- Put communication with your users first
- Plan ahead
- Stay flexible
Want More API Content? Check Out These Posts