What is a security bounty program?
There are different variants, but basically a security bounty program means rewarding people for reporting security issues related to your service. Rewards can range from a T-shirt with your logo on it, to public thanks/fame, to gift vouchers, to small or large cash payments.
Consumer and regulatory attention has focused ever more on the issues of data protection, privacy, and secure online commerce, and security bounty programs are one tool organizations use to ensure their systems are robust.
Security bounty programs have become an established norm for many large firms, open source projects, and even some governments. There are now companies that specialize in operating such programs for large enterprises, and an industry of full and part-time “security researchers”, “bounty hunters” and “ethical hackers” has sprung up.
Our business in context
Before we go further it’s important - especially if you’re new to our blog - to understand the specific context of our business.
We are not (yet!) a large tech firm. Our team is small. We have no external investors, we are entirely funded by the revenues from our customers. We are, as the saying goes, “bootstrapped”.
Our service exists to make using “open” geodata easier for businesses and “prosumers” (academics, etc). If you’re not yet familiar with the benefits of open data we have a guide, but a key attribute is that it is publicly available. The data we have is not unique or proprietary. No one is using our geocoding API for fun; we are a tool they are using to complete a task. The point is, we are not a bank or a fintech firm, nor do we have a big database of consumers using our service.
In addition, all payment processing is done via a 3rd parties (Stripe). We hold no payment credentials from our customers.
In short, we are not a very worthwhile target for hackers.
Nevertheless, we take customer privacy very seriously, and like all EU businesses, we are legally required to abide by GDPR (See our GDPR statement).
So why did we start our security bounty program?
There was no pressing need or expectation from customers or partners that we operate a security bounty program. Still, three years ago we did. Why?
-
First and foremost, we wanted to have confidence that we didn’t have any security issues.
-
We wanted to put on-going pressure on ourselves to follow best practices and not cut any corners. With limited resources and lots to do, it’s always tempting to say “good enough is good enough”. With some aspects of development that is no doubt a reasonable strategy, but not with security.
-
Finally, we want customers/partners/potential future team members to be in no doubt that we are taking security seriously, and are acting responsibly with their data.
And so we launched …
Initially we simply had a security.txt file for researchers to find, which was effective at getting researchers to submit reports, but didn’t make the program transparent to customers and others outside of the security researcher community. So in May of 2020 we launched a page explaining our security bounty program.
How many issues were discovered?
Many, many reports have been submitted. The vast majority of reports (see below in “the bad” section of the learnings) are total nonsense and a waste of everyone’s time.
But not all.
At the bottom of the security bounty page we have a “Hall of Fame” section where we list the different researchers we have paid bounties to by year with a very brief description of the issue. In total it is 20 or so. If memory serves there have also been one or two others who chose not to be listed.
The issues we’ve paid out for are things like
-
We were not hiding server software version numbers in HTTP headers, which is not itself a security issue, but it could make life easier for a potential hacker.
-
Some wiki pages on our public Github repos were editable. Someone could post offensive material, and this might reflect badly on us.
-
On incomplete 2FA setup users were, through an unlikely sequence of actions, able to lock themselves out of their account.
You can see the full list on the security bounty page. It is things that would be annoying for users, or perhaps generate a support request, or things that represent a “code smell” that we are not following best practices. None has been a major issue that got us worried. These are topics that fall squarely into the category of “nice to have”.
Now that even those low-hanging fruit have been picked, we go many weeks without receiving a report, and much longer without awarding any payouts.
There has not (yet!) been a payout in 2023.
How do we decide how much a specific report is worth?
Our general rule is that for every report that prompts us to make a code change or otherwise take action in some way we pay a bounty.
The vast majority of bounties have been for USD 50. A few we have deemed to be slightly more urgent, and for those we have paid USD 100. It is hard for us to justify more for trivial issues like these.
We would gladly pay much higher sums to anyone who made us aware of a legitimate issue.
How much has all this cost us?
I haven’t kept a precise accounting, but looking at the Hall of Fame, we have paid out roughly 25 bounties. In total financial terms, I would guess we have paid out USD 1,250 - 1,500 over three years.
The much bigger cost, by far, has been in our time. As we’ll cover below there are many nonsensical submissions, but still someone has to evaluate them. Unless a report is complete gibberish, we will reply.
What we learned: the good
-
None of the reports we’ve paid out for could be termed a critical bug, and it raises team morale to know we are running a tight ship.
-
The bounty program has worked as a good external factor continually reinforcing the need to stay on top of security. It makes us stay diligent about things like upgrading and following best practices, precisely because we know that if we aren’t it will be discovered by researchers.
-
The financial cost of the program has been negligible. When we started we knew we were doing the basics right, but still it was unclear what people would find and thus what the on-going cost might be. The reality has been much less than our expectations.
What we learned: the bad
- It takes up time I have to admit we probably underestimated the level of distraction this would cause, especially at the beginning. Over time we presumably got added to lists of companies that are willing to pay bounties, and we would see waves and waves of reports come in.
Unfortunately …
-
The vast majority of reports are garbage Despite making claims of being “security researchers”, the vast majority of reports come from very junior developers just running standard security scanners. They often don’t understant the issues they claim to report and most don’t take any time at all to understand the intent of our service. We get many reports that are obviously cut and pasted. Often the company name is wrong or the example URL is for a different site. On the plus side, there are days the reports serve as welcome comic relief because they are so absurdly bad.
-
Many people won’t read. Even though we try spell out very clearly which types of issues we want (and thus are willing to pay bounties for), we get many submissions for things that we explicitly say we don’t care about.
-
Perception/Marketing Value? One of the hopes of the security bounty program was not just that any issues would quickly be found and addressed, but also that customers and potential customers would appreciate our proactive security approach. That may well be the case, but it is very hard to measure in any sort of quantifiable way. As an aside, if you, dear reader, are a customer or potential customers, and appreciate our approach, do please tell us!
What we learned: the ugly.
-
Paying people is not always simple. To pay someone our accounting needs a valid invoice. A shockingly high number of the researchers we have paid out to have not the slightest concept of how to produce such a thing. Now we have a template with what we believe are dead clear instructions. (Protip: search for “basic invoice template”).
-
Many researchers are inexperienced or ignorant. We get people contacting our support email asking if we run a security bounty program. Friend, we all have to start at the beginning, but please learn the basics like what a security.txt file is before you start chasing for business. These types of requests we don’t even answer.
-
Some researchers are annoying or aggressive, insisting they have found some grave exploit. Sometimes you just have to accept other people will be wrong on the internet.
Conclusions / Take-Aways
Overall running our security bounty program has been a positive experience for us, despite some maintenance cost.
Our recommendations
-
Publish a clear list of which reports you will accept and which you won’t.
-
Publish clear guidelines on how you will make payment, what the timeline expectation is, and what sort accounting documentation will be need to make a payment.
-
Prepare for the flood. Especially at the beginning. Have a clear process for dealing with inbound security reports.
-
Make the payment process as simple as possible
-
Have clear requirements of what sort of invoice or similar you need
-
For making the actual transfer of funds to places like India, we use Wise, which we gladly recommend to everyone else - here’s a recommendation link that gets you €500 of free transfers.
-
-
Endurance is needed. In the first year it was not yet clear we would have to wade through a lot of garbage to find a few semi-useful gems, and several times we debated cancelling the program. Over time we learned to be much more efficient. Like any inbound (support, sales) you will need a process to deal with it.
We hope this post helps other founders of bootstrapped tech businesses weigh up the pros and cons of running a security bounty program.
Have questions? Feel free to ping us.
Have a security bug to report? Read - yes, actually read, it will take you about two minutes. You can do it! - our security bounty page before contacting us.
Stay secure everyone,