Avi Das

Home for my code, thoughts and else.

NYC Marathon 2017: Training and Race Report

I ran the NYC marathon this Sunday. On my fourth marathon, I was going for 3:45, came off with a 3:38, personal best by 4 minutes. More than the time, a vanity metric, I was happier about the race execution, doing negative splits, avoiding cramps and bonks/hitting the wall. NYC marathon is a technical and challenging course, but I found it could reward patience and training. It was also an emotional roller coaster for me, NYC being a focal point during majority of my time in the US.

Training

Big part of my marathon was made during the months prior. I have ran marathons before, most recently in Feburary in Austin, so I know my body is capable of handling trials of the 26.2. But I was carrying over my adductor injury from March, and since moving to NYC, its been a slow ramp back up on the miles. Joining the Dashing Whippets in NYC was a great decision, as all my running progress can be attributed to training with groups in Austin, Austin Runner’s Club and Austin Runner’s Meetup. Post June, I had to patiently wait for my speed/endurance to catch up as I stopped running since March. Whippets are a great group, as they are both very competitive and large enough where runners of different paces can have others to run with.

Once summer turned into fall, I was beginning to get the mileage adaptation back up. Besides the Saturday long runs, I worked on Harlem hill repeats on Sundays and speed work with the Whippets on Thursdays. Putting those fast and hilly miles were instrumental to getting myself back into marathon shape. Alongside with Tuesday workout with the whippets, I was able to get my weekly mileage up to 55 early October, more than I’d ever done. However, at this point I had to cut back since the workload was triggering overuse injuries in calves and ankles.

This is what my peak weak mileage looked like

Saturday: 20m long run (Whippets)
Sunday: 8-9m Harlem Hill repeats
Monday: Rest
Tuesday: 12m (Central park Whippets)
Wednesday: 5m easy
Thursday: 10-12m (Speed work on East Side with Whippets)
Friday: Rest

Web Payments by First Principles: Data Architecture

Once you start receiving payments on your site, congratulations! You are likely building something people want. But now you are at the point of having to manage payments data. Developers are generally aware that handling payments data should be done with care, but it is not immediately obvious what the different considerations are. In this blog post, we will go over strategies that you can follow to future proof your payments stack from the point of a fledgeling startup to a mature, stable business.

With storing payments data, there are quite a few considerations. What you should store and shouldn’t. In the event of a security breach on your site, you want your users financial information to be protected. Moreover, you want to architect your data storage for any current and future stakeholder requests. When it comes to payments, there are generally many stakeholders, let’s talk about a handful of them.

You are going to have technical stakeholders: your managers and other product teams who have questions about payments. Business entities that have to report their earnings for filing taxes and reporting earnings to shareholders who will need their data from you. As someone buying/selling online, your data needs to be stored in a way to make sure you don’t break compliance (PCI/SOX etc). Support/operations will be your stakeholders when customers have problems paying and come to you for help. Let’s get into how we can address these asks.

  1. Avoid storing sensitive personal information: Any application sending payments information such as credit card numbers, cvv to their server will have to become PCI compliant. This a financial and logistics burden which you can avoid for the most part by using a gateway provider such as Braintree/Stripe/Adyen. Usually your browser/mobile app will authenticate with the gateway and get back a token, which you can relay back to your server. This removes the danger of accidentally logging payments data, since the only data your server will see is a payments token. Even if you do get data breached, these tokens would not be useful to the attacker. This also removes the need for you to be PCI compliant which is tens of thousands of dollars in yearly expenses. More data you should avoid storing include any plaintext passwords and secret keys, common web best practices.

  2. Freeze request/response from external providers: You need to store every single request/response that you are making to your external providers, ideally in an append-only data storage. One of the common requests that we get in payments is to recreate the transaction as it happened. This is hard to do without storing the data at point in time of the transaction. Moreover, the business logic related to transaction such as taxes, fees calculations also need to be versioned and stored so that you can recreate the transaction at a certain point in time.

  3. Encourage immutability and lower side effects: Similar to the point above, you should never destroy payments or charging data. There are easy ways to archive and hide the data from users. This is very useful for historical financial reporting, triaging potential inefficiencies in your charging/billing process, and dealing with any disputes with your payments provider.

  4. Denormalize and index for searchability: Payments data is generally more write heavy but needs to be stored in a way for ease of triaging. Most payments providers provide unique request ids with their calls, and you should supply your own if that is possible. That way, you can set up bidirectional tracking, so that each individual call to payments is trackable from both sides. Setting indices on those unique ids is helpful for search. If the table is growing too large, it is useful to only keep upto a certain limit in your app and store the rest in a data warehouse such as AWS Redshift or Google Bigquery. The data warehouse strategy also enables you to normalize the data if you want easier access to data in one place and avoid expensive joins.

  5. Prefer cents as units instead of dollars: You can avoid a whole class of floating point bugs by storing in cents and using integers as opposed to float for all your arithmetic. Since floats in computing are really a representation of an infinite number, the can only be approximation and lead to hideous rounding errors. This is a good read on that topic https://stackoverflow.com/questions/3730019/why-not-use-double-or-float-to-represent-currency. Using established tools such as the money gem for currency in ruby is also very useful.

The above strategies will be useful for both internal and external users of your payments stack and help to protect your money and time when it comes to payments data. Please reach out via comments or email with feedback!.

Web Payments by First Principles: Testing

In recent years, payment API providers have made integrating payments much easier than it used to be. Instead of dealing with banks and exchanges, ecommerce apps can integrate with payment gateways that will allow accepting any form of credit cards, and most payment methods such as Apple Pay and PayPal. Large pdfs with instructions manuals are replaced by intelligent documentation sites with walkthroughs and tutorials. Despite that, it is not uncommon to hear developers referring payments as their least favorite part of the development process. Payments integrations are often seen as a necessary evil, to be done once, and hopefully be forgotten thereafter. Often the reasoning is that investing in better payments integration is often not a profit center for companies.

I have worked the last few years in the online payments industry, building APIS, sdks and reliability tools. While payments integration has gotten easier, developers still do make mistakes which are easily avoidable. Here are some of the best practices I would recommend for testing payments in your applications.

  1. Isolate interation between application code and payment gateway in a package: Once an app grows to a certain size, it may have different ways of interacting with payments gateways. You may be accepting recurring payments and accepting webhooks from the payments provider, just in time checkout or interact with point of sales systems. Having your own package that abstracts out interaction with the payments APIs can help centralize all outgoing requests back and forth with the payments API. You can add your own logging and monitoring, stub out the interaction with payments API to have faster unit tests and centralize knowledge about how you serialize and deserialize messages from and to your payments provider.

  2. Sandbox Testing: Most payment API/gateways expose a sandbox environments where you can test out a real integration with the API without moving any money. Ideally your integration tests running continously in Jenkins/Travis/Circle CI should be hitting those endpoints.

  3. Monitoring: You should monitor your sandbox integration as well as your live system. What does the graph of 200s vs 400s HTTP response codes from the payments API look like? Are you getting unexpected 400s? How about 500s? What does the response times look like?

  4. Automated QA: To avoid putting undue stress on your computation and database resources, background tasks are common strategies to do break down calculations for common payments needs such as reporting and analytics. When calculations are done in partial chunks, automated jobs that test whether those calculations have been done properly can reduce a lot of load for your support and developers when something goes wrong midway between a job, or failure.

  5. Negative/Failure Testing: Special card numbers provided by payments providers can help you recreate payment declines due to potential denial from processors for reasons such as not enough funds in account. You may also be able to test for rejections due to fraud and compliance. This helps lower the range of potential unknown errors your site may run into, especially when expanding to new markets or accepting more payment methods.

  6. Live testing: Live testing against payment providers is often tricky, and can led to accounts getting shut down if there is undue load on the API. Despite that, some testing in live is absolutely necessary before you can be confident that on release day, your integration is working as expected.

  7. Test for absence of sentive information: Storing user information such as credit card number or passwords is a very common way of violation of PCI compliance. Regex patterns can be used to make sure that neither your logs nor your database is storing sensitive information.

I intend to write more posts in this series, covering topics such as considerations before and after going live with payments, when scaling up and so on. If you liked this post, please share or comment.

If you have feedback on this blog post or integrating payments, please feel free to reach out!.

Takeaways From MicroConf Starter 2017

With the rise in VC funded startups, there was not a big community for individuals and small teams launching and supporting digital product businesses with their own profits. Rob Walling and Mike Taber noticed that need and created MicroConf, a conference for self-funded software startups.

I was at MicroConf Starter last week. In its 12th year, MicroConf split into Starter and Growth Tiers, the starter edition for people who do not make a full time income from digital products. If you are the kind of person who enjoys taking an idea to a functional product that solves real world problems, MicroConf is a conference where everyone has that shared goal.

Some at MicroConf have launched products and were doing quite well from a digital product business, let it be online courses, software plugins, SAAS etc. There were also number of people who wanted to learn more about find the right idea, product-market fit, sales and marketing.

I really enjoyed the pragmatic voice of the conference, keeping focus on balance. The conference does not shy away from the fact that it is not a easy task to bootstrap software products.

MicroConf has great notes for the whole conference. Instead of trying to go through the whole conference, here are some of my takeaways from MicroConf Starter 2017.

  1. Consistency: Rob Walling emphasized start of the conference that the success of MicroConf will be what these two days can do for the remaining 363 days of the year. Often consistency made the difference in the eventual endurance and success of the product. Josh Doddy’s blog was fairly dormant for the first 12-14 months but peaking exponentially near its current runtime of 18 months. Mastermind groups were mentioned as a great way for a group of people who help each other stay on track.

  2. Finding an idea worth building: Multiple speakers mentioned the need to take a hard look at your stocks and assets. What questions do people keep asking you? What are you passionate about that other people find boring? What would you from 6 months ago find valuable? All these were from Ben Orenstein’s talk, one of my favorite at the conference. Patrick Mckenzie also touched on the same topic, to double down on what you do very well already and what the market already buys from you. Justin Jackson mentioned the need to find the groups you are best equipped to serve, and to research the audience and find ideas rather than thinking in your own head what the problem could be. Mike Taber also emphasized focusing if you are in fact the right person to be building that product.

  3. When to launch: An MVP should solve a well defined problem, not solve a portion of it or solve every possible iteration of the problem. However, the lack of polish is intentional to see how much inconvenience the customers would endure to solve their problem. Justin Jackson had a great point that an MVP should be the smallest product you could build to disprove a hypothesis. It was interesting to see multiple speakers mentioning the importance of putting your face right by your product, to encourage trust and take responsibility of what you are delivering.

  4. On user acquisition: Probably the biggest concern of fledgling products, user acquisition/outreach had dedicated talks. Some of the key points where to focus on conversations with users and doubling down on a few approaches e.g. SEO, Content, Ads rather than throwing in the kitchen sink. Looking for integrations with other products by forming partnerships was a common theme. Key questions to ask users were to ask how they were solving the problem today, what they have tried or ask for introductions to someone with that problem. In the beginning, unscalable strategies such as concierge onboarding are useful, specially for SAAS products.

  5. Getting results: Users are the best signals here, and if you have to chase people down to use your products, it may not be solving a real problem. Google analytics charts showing growth and conversion rates were part of almost every presentation. At the same time, reading those charts can be a hard story, often showing charts recovering from a flatline or decline to eventual success because the founders believed enough to carry on. Its hard to think convictions to be infallible in the face of data however, and sometimes it is time to give up.

What Makes a Great Programming Tutorial?

The Internet started as a publishing medium. It excels at enabling people to share their unique gifts. An amazing amount of content gets put out on the web everyday, far beyond someone can read in a lifetime. Massive amount of information also means information overload.

In software, it is common to use web tutorials to supplement one’s learning of a particular material. Following guidelines of someone who has already done it can really fast track development. Tutorials appear in various forms in the web. Some of the common formats are long form blogposts, videos and series of email newsletters. Some enjoy the personal touch that videos can offer; others enjoy being able to quickly skim a blogpost.

When you want to put your hard earned knowledge and valuable time into writing a tutorial, there are questions worth thinking about. Does the tutorial cater to its intended audience? Will it reach them? Can someone quickly skim the content and still get value? Is there a way to measure the effectiveness of online tutorials?

Survey Questions

To look for answers, I asked a carefully selected group of 150 people about their preference of tutorials. The focus was on software engineers/designers/entrepreneurs due to my familiarity and experience with the field. What emerged from their responses gives a blueprint for creating great online resources.

  1. Real time feedback: If exercises or examples accompany a tutorial, multiple survey responders emphasized the need to check the user’s responses and provide interactive feedback to the user to guide them to the solution. Good examples are checkpoint multiple choice questions during Coursera/Udacity courses that you must complete to move forward with the course.

  2. Follow up: Several responders emphasized the need to provide contact information or other means to reach out to the author once they went through a tutorial. Providing a comments section or your email/twitter handle are great mediums for a reader to follow up.

  3. Address a concrete problem: Staying focused of a specific problem gives a tutorial depth. A differentiator can be to classify for the user whether the focus is academic and structured vs. a blogpost focused on solving a particular problem right now.

  4. Working examples: Interactive, working examples close to the content that builds on top of each other makes following along simpler. Michael Hartl’s Ruby on Rails tutorials came up as an example of detailed, comprehensive tutorials.

  5. Advanced user tutorials: Several user’s pointed out the need for tutorials that addresses advanced content. Making expectations clear about experience levels of intended audiences can be a huge positive for tutorials. Ray Wenderlich’s iOS tutorials are good examples of the detail and level of research and understanding that can happen before putting out content on the web.

10 Things I Learnt Going From 10k to a Marathon in 2015

If I were to think about running a 26.2 mile race at the start of 2015, the overwhelming feeling would be one of fear. I have only ever ran a 10k before and just signed up for the Austin 2015 half Marathon, my first ever half. The prospect of running twice the distance still seemed far away though. Flash forward to October 18th 2015, I finished my first Marathon with a 3:49:00 time. As I look back on this year, I wanted to put together some of my realizations during the whole process. 

  1. Running is a privilege: Living somewhere where I can run safely, have trails to run on, be in good health to run are all privileges to be thankful for. Growing up in the developing world meant that it was hard pressed to find opportunities to be involved in outdoor activites. Having the time and space to exercise is a luxury that needed to be earned. I never ran in high school, and by the time I graduated college I could not run longer than 5k. Having the time to run, being in US where running is very much part of the culture has been a huge contributor to my running progress.

  2. Joining a running group is one of the best decisions you can make as a beginning runner: Training with people better than you to improve is not unique to running. Start of 2015, I had no plans of running a marathon. In April, I joined the Austin Runners Meetup (ARM). In training long runs with ARM, I was able to build up the endurance for longer runs which made the progression to a Marathon much easier mentally. Training with other runners can definitely help you maintain the habit of running as well as improve your form and performance. Moreover, I found a new community of great people which has been very rewarding.

  3. Tying running with other activities you enjoy can make running much more consistent: In Charles Duhigg’s Power of Habit, he talks about the cue-action-reward pattern that most habits follow. Being aware of that pattern can help in building running as a habit. When run’s are followed by a delicious breakfast, you have something to look forward to. Trance music and podcasts help me maintain the flow during running. Travel is one of my favorite things and going to a new city for a race is something I eagerly look forward to.

  4. Running is a blissful release from life’s distractions:

    “All I do is keep on running in my own cozy, homemade void, my own nostalgic silence. And this is a pretty wonderful thing.”  ― Haruki Murakami, What I Talk About When I Talk About Running“`

    Distractions are part and parcel of our lives as more form factors emerge competing for our limited attention span. Often, this results in us not being aware of the passage of time. Running has been a great antidote to that problem for me. When running is effortful, you have to concentrate on the activity at hand and your entire focus is on the present moment. Long runs offer the prospect of seeing places and neighborhoods that you would not frequent otherwise. A slight breeze on a scorching summer’s day has never been more enjoyable.

  5. Once running is a habit, a chore becomes a craving: When you start out running, it can be something you dread on your calendar. There is really no way to get past this without sustained practice, lowering the cognitive load with group associations and reward mechanisms. If you continue though, you realize at one point that you start craving the runs. Don’t get me wrong, it still requires a lot of mental effort to get up at 5:30am for that 20 mile run you need to do for Marathon training. But something about the combination of endorphins, habits built during running and seeing your friends out there on the trail can change running into something that you look forward to.

Coffeehouse Coworkers

I had a problem.

It is fun to start side projects, but not always the easiest to stay committed and finish. Coffee has powered more projects than one can count. What also helps to stay focused and motivated is the company of like-minded individuals. Coffee and friends results in more and better products.

What can one do to reach more of those people?

Coworking spaces are hot in this economy. WeWork have recently stepped into the $10 billion club. In Austin, Capital Factory, Chicon collective, General Assembly and Link are just some of the spaces ranging from Accelerator/Incubator to just rentable office spaces. However, they are much more fitted for startups working out of a coworking space. Moreover, one spot can get boring pretty quickly.

What if you are a remote worker or work on side projects and wish there was a community you could cowork/share with?

In Austin, there are a few options for that as well. There is a great group called Cafe Bedouins who meet Tuesdays at 7pm in Houndstooth cafe to work on projects. I had a great time there, but thought why would this need to happen only on a particular weekday on a particular time? Weekends are often the times when side projects gets the attention anyway. By a stroke of luck, I ran into Adam Coulon at Cafe Bedouins, who is also really enthusiastic about coworking, and shared the same problem.

We considered platforms where people could spontaneously decide to meet and cowork on projects. Slack was the clear winner. It is really easy to set up, and it has spread like wildfire so people tend to be familiar with the product. Without futher ado we present Coffeehouse Coworkers, made with rauchg’s excellent slackin pluigin. It’s a slack channel for people to find others and decide on places to cowork!

At the highest level, we love products and want their to be more finished products out there. It could be a blogpost, design concept, open source software or your consulting business. We think that better products happen in collaboration with like minded people. This may not be everyone’s cup of tea (or coffee rather), but our hope is to enable some people to optimize their workflow in a low commitment way.

If any of this interests you, join us at Coffeehouse Coworkers. Right now, the members are mostly Austin based and in tech since that is our current demographic, but it does not really have to be limited to that. Austin is a great place to start since it has a lot of independent workers and great coffeeshops.

Happy coworking!

Pycon Canada 2015 II: Python for Reliably Delivering Cross Platform Products

This is the second part of the two series blog post about my talk at PyCon Canada. Here is the first part.

Proposal:

Are you part of a team responsible for delivering cross platform products? End to end automated testing and communicating effectively are important when your project depends on multiple teams spread across functional domains. At Braintree/PayPal, we worked on a framework to reliably ship developer facing products. We will go over using BDD with Behave to describe test scenarios that speaks to both product and engineering, using Appium for mobile automation, and ElasicSearch and Kibana for storage and visualization of test results. You will see how the breadth of packages available on PyPI, the flexibility and ease of Python helped a team of developers whose core competencies were not in Python to collaborate on a common ground.

Slides:

Video:

Youtube link

Github Repo

Reliability Demo