Our Move to Stripe and Salesforce for Donations

The Texas Tribune is a nonprofit, and as a nonprofit, memberships and donations are very important to us. The code powering the pages for becoming a Texas Tribune member, renewing a membership, and making one-time donations used to live inside of the main Texas Tribune codebase. As we mentioned in our deletions sprint blog post, in June 2015 we moved this code out of our main codebase.

Why did we move our membership and donations pages out of our codebase? We’ve been working to take apart our monolithic codebase that’s working hard to do everything, and split it up into smaller elements, with some work outsourced to third-party services. A smaller main codebase means that it’s less brittle, and that an issue in one area of the site doesn’t negatively affect another area of the site. In addition, making our code more modular makes it easier to manage, easier to modify, and easier to conceptualize about. Thoughtfully choosing focused tools made for each job also means we’re using each piece of code for what it’s best at and not forcing the code to do what it doesn’t want to.

We also didn’t want to be responsible for taking credit cards on our site anymore; it was a better choice for us and our small four-person team to outsource that work to a third-party payment processor that can spend more time focused on security and best practices than we can. In addition, we liked that our ability to accept memberships and donations wouldn’t be affected if other parts of our site experienced issues.

We looked into a few options for what to build with. We chatted with Voice of San Diego (VOSD), a fellow member-based nonprofit news organization, about what they used. VOSD and MinnPost adopted Givalike because it played nicely with Salesforce and ostensibly catered to nonprofit organizations like ours. We had similar goals that we wanted to fulfill with our payment processor, so it seemed like a great fit for us, and we knew if we used it, we’d be in good company.

We also needed to decide what we would use to build the membership and donation pages. We’d previously used Middleman, a static site generator, to build a site to celebrate the Tribune’s 5-year anniversary and had enjoyed working with it. It was fast to develop with, and it provided everything we’d need. Since we knew that most of the information on our membership and donations pages would be static, we didn’t want to overcomplicate it. And the parts of the site that would change, we could set up to update with Vox’s Middleman and Google Drive gem. This would allow our membership team, as well as other colleagues who might want to update text on the page but wouldn’t be committing code to GitHub and redeploying the app, to make changes without much help from our team.

At this point, it had begun to become clear to us that it wasn’t just us at the Tribune: nonprofits’ need for membership and donations pages like ours was real.

When we learned that a Knight-Mozilla OpenNews Code Convening would be taking place soon in Portland, Oregon, we felt that taking a pass at making the work we’d done to build our donations site open source and more friendly to other organizations’ customizations would be a great project. The wonderful folks at OpenNews agreed. Kathryn flew to Portland and spent a few days putting together DonationBuilder.

The sun was shining and birds were singing. We were feeling good and accepting donations through our new membership and donations site and Givalike. Everything seemed to be going along fine.

Unfortunately, however, the honeymoon with Givalike as our payment processor was short lived. We started encountering problems with Givalike within a month or two, such as transaction failures and subsequent lackluster responses to those failures, in addition to other concerns. Facilitated by our shared Salesforce consultants, Anna Crotty and Mark Olinger, our three organizations (The Texas Tribune, VOSD, and MinnPost) began meeting regularly to join together and prioritize our issues so we could present them to Givalike as a group. Despite combining forces and speaking with one voice, we did not get much traction with Givalike. We were feeling frustrated, and we also knew we couldn’t accept this and the effect it was having on our members’ and readers’ experiences as a part of our community. It was time to begin researching a better way.

We started looking into an alternative at the end of June 2015. At first, we poked at it a bit, but didn’t start quickly developing a switch. Then we were notified that on August 5, 2015, only a little over a month later, Givalike would be transferring operations to another service, Donatic. This revelation lit a fire under us about switching to Stripe–what we’d been considering a likely future event had become an imminent need.

Although we needed to move fast, we wanted to be sure to do thorough research on a switch, so we looked into other options beyond Stripe. We weren’t excited about the fees that came along with other services–usually they were in the 5 percent range, which wasn’t too appealing when compared to Stripe’s 2.9 percent. Using Stripe would require writing some custom integration code, but with its lower fees, it was in line with the rest of the bare-bones payment processors we knew about. Sweetening the deal even more was the fact that Stripe’s API is highly regarded among developers, and we’d heard that it was enjoyable to work with. As we began working with it, it was clear that it was all true: Stripe’s API was well documented and a breeze to set up and work with.

As we were building out our Stripe integration to switch over as soon as we could, we received another notification on February 2, 2016, informing us that Donatic would be ceasing operations on March 31, 2016. This news only confirmed that we’d made the correct decision.

We considered doing the integration work in Salesforce itself but decided against that approach because we didn’t have anyone in our newsroom who knows Salesforce’s proprietary programming language. We were much more comfortable using Salesforce’s REST API and calling it from Python, a language with which we’re fluent. Using Python also meant that we’d have a room full of people who could author and maintain the code, and we wouldn’t be relying on a likely-quite-expensive outside consultant to build or maintain it. So we fired up a Flask app and started building.

We chose Heroku as the deployment platform because it insulates us from many of the unpleasantries of running our own infrastructure. With just a few clicks we can deploy and run our application. And at around $25-50/month per organization, the price was right.

In Salesforce, we needed to map fields in Stripe to the appropriate Salesforce field, which required creating a few custom fields on the recurring donation object (part of Nonprofit Starter Pack 3) and the opportunity object.  NPSP has a configuration tab that allows an admin to map fields on the recurring donation to the related opportunities, so that the Stripe app only creates one record for a recurring donation.  The changes required in Salesforce were put into a package to allow each organization to install them all quickly.

The integration is up and running smoothly now, and was overall a successful example of collaboration and teamwork. We at The Texas Tribune (mostly Daniel Craigmile, Kathryn Beaty, and Anna Crotty) built out the foundation for all three organizations’ integrations.

There are a few things we’d do differently, of course; with every project there are always a few things. Each integration had to be customized independently for the needs of each organization. If we had more time, we would have pushed more of that customization from code into configuration to make it easier for each organization to adapt to their specific needs. Even though all three organizations are using Stripe and Salesforce, each organization has very different Salesforce customizations, which isn’t ideal.  

We hope that our story and what the three organizations have built between us can be of use to other organizations tackling similar challenges.

Salesforce-Stripe integration: https://bitbucket.org/texastribune/salesforce-stripe/
Donations app: https://github.com/texastribune/donations-app


Deletions Sprint

Deletion celebration tweet
Deletion celebration tweet

Over 15,000 and still counting. That’s how many lines of code we removed from the Tribune codebase in June of 2015. That’s about 3% of the codebase, which remains over 455,000 lines strong, according to a Sublime Text regex search.

How did we get here? Our team works in two-week sprints, and we sometimes give our sprints themes, which means all three of us work on tasks focused around a single goal, like performance. We’d had deletion of legacy code hanging around on our list for awhile. The Tribune has been around for over five years now, so there was quite a bit of code that the project had evolved beyond. We have code to be deleted detailed in Github issues and Basecamp lists. Some of this code has been a part of the site since as early as October 2009!

When you tell non-programmer colleagues that you want to spend a few weeks focusing on deleting code and lessening your technical debt, be prepared for a blank look. Then you’ll have to translate “code deletion” and “less technical debt” into an explanation of why it’s important to you (and to them, and the rest of the organization) that you delete that code.

I like to use the analogy of a house; if your rooms are cluttered with unused items, it’ll take you a lot longer to navigate through and get things done. If you have a nice clean house, however, you’ll be able to, for example, easily find your way to the kitchen and cook up a delicious dinner. Same with a codebase. If there’s unused code hanging around, it’ll take you longer to wade through it to see what it’s really doing and add a new feature or improve the existing ones. But when the codebase is clean and shiny, the new feature is easier to add in and can be built faster.

So recently, after explaining why cleaning up legacy code is beneficial to everyone within earshot of our corner of the newsroom, and reminding everyone multiple times that’s what we’d be focusing on for our sprint, we finally got to roll up our sleeves and get down to the business of deletion.

This summer was the perfect time for us to clean up the codebase, too, before our newest team member, Liam Andrew, joined us. This way, Liam wasn’t distracted by unused code, which eased his period of orientation with the codebase. We’re also gearing up for a few big projects on the horizon, so this helps us get ready to attack those. Our codebase feels fresh and clean now; although it’s only about three percent, it feels like more. Immersing ourselves in code deletion has also been a positive because it’s made us better programmers. When you’ve been in the weeds deleting unused code, it makes you approach adding new code with a more complex perspective and refactor code as you work.

This is how light it feels
This is how light it feels

Much of the code that we deleted was replaced with something else that provided a similar, but improved, functionality. For example, we removed the code powering our previous donation page when we went live with our new donations app where people can become members and support our work. We also removed our homegrown Twitter app taking up a large amount of database tables that were powering Twitter widgets on our site, and replaced them with the widgets provided by Twitter, removing the burden on our database and codebase and placing it onto theirs. In addition, we removed the paywall that we had been housing inside our code for our premium Texas Weekly product and replaced it with a third-party paywall, Tinypass.

Some things we archived and then deleted the code powering them. From this process of deletion, we’ve come to appreciate more deeply how important it is to think through the process for sunsetting a project once it’s no longer relevant. This is especially important for news organizations where most stories are at least somewhat time-sensitive. Otherwise, projects will hang around forever and ever and ever and related code will break. Sunset plans allow you to create a pristine archive of your organization’s work, and they’ll ensure old projects don’t keep you reliant on requirements and legacy code that could hold you back.

The beauty of having a sunset plan
The beauty of having a sunset plan

When you’re removing code from a large project, especially code that was written by a previous developer, it can be scary to delete a significant number of lines. What if you forget to delete something related and leave behind orphaned code? Most frighteningly, what if you miss something dependent on the code you’re deleting and break parts of your site?

Our test suite gave us some comfort when it came to removing this code. So did deploying and testing out the code in our staging environment, which closely mimics our production site. We also followed our policy of always having another team member comb through our code changes. That got-your-back help of having another pair of eyes on your work can be integral to catching all the details.

We’ve also stepped up our documentation game. We’d already started fattening up our internal wikipedia prior to all these deletions, but the deletions definitely kept the momentum going. I think of looking at legacy code like an archeological dig; the more of the bones and tools you have, the better you can understand what you’re looking at. And if you understand how code was used, you’ll know what you can safely remove when that code’s no longer needed.

Don't let your codebase be as mysterious as Machu Picchu
Don’t let your codebase be as mysterious as Machu Picchu

Have you recently deleted some code from your organization’s codebase, or another project you’re working on? Do you have some code you’ve been meaning to remove but haven’t found the time and space to get around to it yet? We’d love to hear the story of your deletions, any challenges you came across, and how it feels now if you’ve already deleted the code and lessened your technical debt!

Hello world

Bees on Honeycomb
Bees on honeycomb made of hexagons

Welcome to The Texas Tribune platforms team tech blog! We’re a lean, four-person team comprised of Director of Technology Amanda Krauss, System Architect Daniel Craigmile, Developer Kathryn Beaty, and Developer Liam Andrew.

We support the efforts of the fantastic Texas Tribune editorial, events, marketing, membership, and sponsorship teams. We work on the CMS and websites for The Texas Tribune and TribTalk, both of which are Django apps. We integrate third-party services such as Eventbrite and Mailchimp into the site. Since we have a small team, we all wear many hats and have a hand in everything from maintaining the servers and databases on the backend to building with tools like Sass, JavaScript, and Grunt on the front end.

Why is our blog called Notes from the Hexagon? A hexagonal desk sits in the center of our corner of the newsroom, and when we’re not typing away at our standing desks, we’re sitting around its six sides with our laptops. A few of our colleagues started calling our space The Hexagon, and we embraced the shape as part of our team identity. We recently got hexagonal power plugins. And we like that hexagons are found in nature in bees’ honeycombs, the shape of Petoskey stones, and more. It’s safe to say that we love hexagons, and so it was only natural to name our blog after them.

Hexagon Power
Our hexagonal power plugins

In our blog, we’ll take turns writing posts sharing insight into the projects we’re working on, the challenges we’re facing, the successes we’ve had, and more. We figure that if we’ve come across a challenge, chances are someone else is also trying to find their way through that same challenge. Plus, it only makes successes sweeter to share them!

We’d love to hear your feedback on our blog, so please leave a comment, tweet at us, or send an email. Thanks, and we hope you enjoy Notes from the Hexagon.