Avi Das

Home for my code, thoughts and else.

Core Tenants of Highly Effective Software Teams

This blogpost is my thoughts only and does not necessarily represent the positions of current or past employers.

We don’t build software in a vacuum. Software involves people. Beyond organizations of a handful of people, hierarchy is beneficial. We get teams, commonly with engineering manager/lead, product manager, designers and engineers. What becomes crucial for the software and the product delivered then is the effectiveness of the team. Throughout my career in the industry and being part of many teams in different circumstances, I have started noticing some key patterns that really drives standout results in teams.

  1. Believing in a common cause: The single biggest observation is that when a team of people believe in a common mission, they produce outsized returns. The most effective teams I have worked in all had a strong belief that they was a reason for the work they were doing. This also aids inter team collaboration over inter team competition, with teams often investing in tooling that makes the whole team better.

    Engineering leaders can play a key role here to frame a compelling mission for the team. Hiring for the right role also becomes super important as a highly motivated individual in a role can be 2-10x more effective than someone unmotivated with similar ability. Having a competitor or a common enemy is great since we are predisposed to bond over defending ourselves from common enemies.

  2. Psychological Safety: Google’s Project Aristotle studied 180 teams over two years and came to the conclusion that psycological safety was the best signal for how effective a team is. How comfortable do people in the team feel to share vulnerability without fear of retribution? How comfortable do people feel asking questions without fear of asking something silly or share ideas without fear of being shut down without listening? Team’s with high levels of psychological safety can have conflicts, but can deal with them in a mature way, being able to separate disagreement about ideas from disagreement with people.

    For more senior engineers/technical leaders, this is crucially important since they are in a position to determine this culture for the team. Forming strong personal relationships with the team can be really valuable for fostering safety within the team. People like their leaders to be human, and admitting your own fallibility is a great way to form trust with team members.

  3. Diversity of Thought: Diversity is a word that is commonly heard in the tech industry, and for good reason. Having diversity of people is a proven way to achieving diversity of thought, which is just one of the reasons why we must invest in software communities of women and minorities. Inclusiveness is one of the key pillars of psychological safety in a team, building on from the last section. Moreover, when software is aimed at global audience, but the team is homogeneous, it is easy to be fooled that a wide audience will get their needs met.

    Even teams of experienced contributors can fall prey to atrophy and decay, without fresh ideas so common in upstarts. A team of really excited newer developers may not realize that in balance lies the key to long term personal, team and product success. Diversity of experience in a team helps to avoid these common traps.

    Finally, cross functional teams can be more effective than teams exclusively focusing on frontend/backend/mobile. Recognizing the individual contributor’s interest in user experience/security/governance etc and enabling space for that one of the most enabling things an engineer leader can do.

  4. Growth and Ownership: It is immensely gratifying for people to feel that they are growing, and knowing that they are playing a role in the growth of others. When team members feel confident about the path in front of them can still have challenges, they are far less likely to be unmotivated and plateau. This is big for retention, since job changes frequently are a result of people feeling stuck and needing to make a change. It is costly to replace engineers, especially ones already trained and performing well in their role.

    A key intrinsic motivator for many is the feeling of ownership. Being able to really sink their teeth into a hard problem and come up with something they are proud of. Teams where people really believed that they have strong ownership of the product also care more about the end users experience, resulting in a better product.

    As engineering leadership, one of the best signals of good management is to have clarity in career ladders and promote the right people. A bias for people who make others around them better can be healthy. It is my experience that promotions should rarely come as a surprise to the individual or the team. Demonstrated investment in people as future leaders is also a major indication of a company’s belief in their people, sending them to conferences, training and giving license for creativity.

  5. Work Environment: This is a controversial one, but I do believe that companies today have bought way too much into the open office movement. While a return to cubicles does not feel desirable, dedicated interruption free zones (both space and time) are essential for good software. A chaotic office environment can also mean chaos in your codebase.

    Debates range whether standing desk or sitting is better, however many monitors are necessary. My belief here is that the team should be colocated but individuals should be empowered to find the best working situation for the track of work they are in. I have personally found that standing keeps me on my toes, making it great for lots of small tasks, whereas sitting is best for tasks that need deep thought.

When things fall apart

We do not live in a perfect world. Recessions, unexpected downsizing, market competition and many other forces can impact access to resources which could result in ways in a group of people come in to work together and stop working together. Lot of us have all worked in a team where that magic of a great team existed, and the team achieved things together what could not be achieved by individuals. It is important for us to be thinking with intention and purpose and help each other build and find teams to discover that magic.

Jersey Half Marathon 2018: Race Report

“Gatoradeee!! Water!” The sudden enthusiastic cheer after a period of silence was hard to miss. Looking up, I saw the 10k marker. I looked at my watch and I was about to PR a 10k. Except that I was not running a 10k. I was running a half Marathon and with more than half of the race still ahead of me, this was bad news.

Post NYC marathon last November, I was happy to have run my goal race in a good time. I knew I was hitting my upper limits with the Marathon, and without focus on shorter distances, I would not get faster. I focused religiously on the Tuesday tempo and Thursday speed workouts with the Dashing Whippets central park group. It was inspiring to see the people training for Boston Marathon putting in incredible work during some difficult months.

This season’s training posed many challenges, primarily freezing temperature, snowstorms, breakups. Every time I stepped out the door and breathed in the icy air into my lunges, everything inside me wanted to get back inside and wrap myself in a blanket. But the workouts had a way of enforcing discipline into my life. For Saturdays, I made the commitment to keep showing up and sometimes challenge myself by going with a slightly faster group.

The group kept me going. If everyone else have no problems showing up and putting in the work in dark and cold, I have no excuses not to. When I was in the pain cave during the Jersey Half, that was what going though my head. I am in the deep end, but I owed my 600 mile training cycle a good performance and take responsibility for strategic mistakes early in the race.

As I realized my mess up during the Jersey half, I realized I needed a baseline pace or I would fade. My inner voice said don’t fade, every second counts. So I found couple runners putting down 7:15 miles and making it look like cakewalk. Later I learnt that they were running the marathon. Talking to them helped me get some boost. After that, I found another pace group, and hung with them all the way to the end.

The value of this race as a developing runner is that it answers some lingering questions. How fast am I? Am I capable of taking my progress in training and convert it on race day? Am I training with the right group of people or just tagging along with faster runners for dear life. That’s what this half will mean for me, that I have improved this training cycle and while my strategy and mental game needs work, I can enjoy the following week knowing I made progress. Progress where I used to think there is no way I can hold a 7:09 pace for 13 miles but there is now data to prove otherwise.

So that was the Jersey Half. I am very happy with the result having converted a 10 minute PR. However, I would like to get there next time more gracefully and not feel totally wrecked at the end.

Lower Degrees of Separation With End Users

When working in software, one way to look at our profession is to say that we take architecture docs or designs and make code out of it. After years in the industry, we are trusted to come up with the architecture docs and work with a team to deliver the software. This absolves ourselves of responsibility in a way since even if the product fails, at least our code and systems were great. Companies today, however, are starting to see the limitations of software engineers being removed from the product decision making process.

I think we should reframe the problem: it is rather our responsibility as software engineers to ask, how many degrees of separation does it exist between us and the end user? Ideally, the end user would be the person paying for our service, although this gets more complicated certainly by ad funded or venture funded software. The exercise could involve us asking, what would it take to reach 10 users of our software? Would we have to go through our product manager, who then talks to the account manager or product support? These are likely the folks currently dealing with customer calls when our software bresks and waiting for the Zendesk tickets to be picked from the queue.

Who we are “engineering” for is a question we need to frequently ask ourselves. We should strive to be in environments where we are aware of our degree of separation and look for ways to cut down that separation. Without that frame, we can only have vague ideas of what the code we write is leading to, and end of the day limits the impact we can have.

It should also not always be the product manager’s job to always acting as the liason to translate user needs to us. When we are aware of user needs, it enables us to be proactive: to avoid that shortcut when building, or deal with that performance bottleneck early before it becomes a problem. We can also free the up the product manager to pursue broader goals such as product vision, market and competitive landscape analysis, etc.

Tomorrow, when you get to work, ask yourself that question. Do you know who your users are and how they use your product? How many degrees of separation would you have to navigate to find that answer? If you are not comfortable with the answer, maybe you can think of a way to change that.

Disclaimer: Thoughts expressed in the article are mine only, and does not represent the positions of current or past employers.

NYC Marathon 2017: Training and Race Report

I ran the NYC marathon this Sunday. On my fourth marathon, I was going for 3:45, came off with a 3:38, personal best by 4 minutes. More than the time, a vanity metric, I was happier about the race execution, doing negative splits, avoiding cramps and bonks/hitting the wall. NYC marathon is a technical and challenging course, but I found it could reward patience and training. It was also an emotional roller coaster for me, NYC being a focal point during majority of my time in the US.

Training

Big part of my marathon was made during the months prior. I have ran marathons before, most recently in Feburary in Austin, so I know my body is capable of handling trials of the 26.2. But I was carrying over my adductor injury from March, and since moving to NYC, its been a slow ramp back up on the miles. Joining the Dashing Whippets in NYC was a great decision, as all my running progress can be attributed to training with groups in Austin, Austin Runner’s Club and Austin Runner’s Meetup. Post June, I had to patiently wait for my speed/endurance to catch up as I stopped running since March. Whippets are a great group, as they are both very competitive and large enough where runners of different paces can have others to run with.

Once summer turned into fall, I was beginning to get the mileage adaptation back up. Besides the Saturday long runs, I worked on Harlem hill repeats on Sundays and speed work with the Whippets on Thursdays. Putting those fast and hilly miles were instrumental to getting myself back into marathon shape. Alongside with Tuesday workout with the whippets, I was able to get my weekly mileage up to 55 early October, more than I’d ever done. However, at this point I had to cut back since the workload was triggering overuse injuries in calves and ankles.

This is what my peak weak mileage looked like

Saturday: 20m long run (Whippets)
Sunday: 8-9m Harlem Hill repeats
Monday: Rest
Tuesday: 12m (Central park Whippets)
Wednesday: 5m easy
Thursday: 10-12m (Speed work on East Side with Whippets)
Friday: Rest

Web Payments by First Principles: Data Architecture

Once you start receiving payments on your site, congratulations! You are likely building something people want. But now you are at the point of having to manage payments data. Developers are generally aware that handling payments data should be done with care, but it is not immediately obvious what the different considerations are. In this blog post, we will go over strategies that you can follow to future proof your payments stack from the point of a fledgeling startup to a mature, stable business.

With storing payments data, there are quite a few considerations. What you should store and shouldn’t. In the event of a security breach on your site, you want your users financial information to be protected. Moreover, you want to architect your data storage for any current and future stakeholder requests. When it comes to payments, there are generally many stakeholders, let’s talk about a handful of them.

You are going to have technical stakeholders: your managers and other product teams who have questions about payments. Business entities that have to report their earnings for filing taxes and reporting earnings to shareholders who will need their data from you. As someone buying/selling online, your data needs to be stored in a way to make sure you don’t break compliance (PCI/SOX etc). Support/operations will be your stakeholders when customers have problems paying and come to you for help. Let’s get into how we can address these asks.

  1. Avoid storing sensitive personal information: Any application sending payments information such as credit card numbers, cvv to their server will have to become PCI compliant. This a financial and logistics burden which you can avoid for the most part by using a gateway provider such as Braintree/Stripe/Adyen. Usually your browser/mobile app will authenticate with the gateway and get back a token, which you can relay back to your server. This removes the danger of accidentally logging payments data, since the only data your server will see is a payments token. Even if you do get data breached, these tokens would not be useful to the attacker. This also removes the need for you to be PCI compliant which is tens of thousands of dollars in yearly expenses. More data you should avoid storing include any plaintext passwords and secret keys, common web best practices.

  2. Freeze request/response from external providers: You need to store every single request/response that you are making to your external providers, ideally in an append-only data storage. One of the common requests that we get in payments is to recreate the transaction as it happened. This is hard to do without storing the data at point in time of the transaction. Moreover, the business logic related to transaction such as taxes, fees calculations also need to be versioned and stored so that you can recreate the transaction at a certain point in time.

  3. Encourage immutability and lower side effects: Similar to the point above, you should never destroy payments or charging data. There are easy ways to archive and hide the data from users. This is very useful for historical financial reporting, triaging potential inefficiencies in your charging/billing process, and dealing with any disputes with your payments provider.

  4. Denormalize and index for searchability: Payments data is generally more write heavy but needs to be stored in a way for ease of triaging. Most payments providers provide unique request ids with their calls, and you should supply your own if that is possible. That way, you can set up bidirectional tracking, so that each individual call to payments is trackable from both sides. Setting indices on those unique ids is helpful for search. If the table is growing too large, it is useful to only keep upto a certain limit in your app and store the rest in a data warehouse such as AWS Redshift or Google Bigquery. The data warehouse strategy also enables you to normalize the data if you want easier access to data in one place and avoid expensive joins.

  5. Prefer cents as units instead of dollars: You can avoid a whole class of floating point bugs by storing in cents and using integers as opposed to float for all your arithmetic. Since floats in computing are really a representation of an infinite number, the can only be approximation and lead to hideous rounding errors. This is a good read on that topic https://stackoverflow.com/questions/3730019/why-not-use-double-or-float-to-represent-currency. Using established tools such as the money gem for currency in ruby is also very useful.

The above strategies will be useful for both internal and external users of your payments stack and help to protect your money and time when it comes to payments data. Please reach out via comments or email with feedback!.

Web Payments by First Principles: Testing

In recent years, payment API providers have made integrating payments much easier than it used to be. Instead of dealing with banks and exchanges, ecommerce apps can integrate with payment gateways that will allow accepting any form of credit cards, and most payment methods such as Apple Pay and PayPal. Large pdfs with instructions manuals are replaced by intelligent documentation sites with walkthroughs and tutorials. Despite that, it is not uncommon to hear developers referring payments as their least favorite part of the development process. Payments integrations are often seen as a necessary evil, to be done once, and hopefully be forgotten thereafter. Often the reasoning is that investing in better payments integration is often not a profit center for companies.

I have worked the last few years in the online payments industry, building APIS, sdks and reliability tools. While payments integration has gotten easier, developers still do make mistakes which are easily avoidable. Here are some of the best practices I would recommend for testing payments in your applications.

  1. Isolate interation between application code and payment gateway in a package: Once an app grows to a certain size, it may have different ways of interacting with payments gateways. You may be accepting recurring payments and accepting webhooks from the payments provider, just in time checkout or interact with point of sales systems. Having your own package that abstracts out interaction with the payments APIs can help centralize all outgoing requests back and forth with the payments API. You can add your own logging and monitoring, stub out the interaction with payments API to have faster unit tests and centralize knowledge about how you serialize and deserialize messages from and to your payments provider.

  2. Sandbox Testing: Most payment API/gateways expose a sandbox environments where you can test out a real integration with the API without moving any money. Ideally your integration tests running continously in Jenkins/Travis/Circle CI should be hitting those endpoints.

  3. Monitoring: You should monitor your sandbox integration as well as your live system. What does the graph of 200s vs 400s HTTP response codes from the payments API look like? Are you getting unexpected 400s? How about 500s? What does the response times look like?

  4. Automated QA: To avoid putting undue stress on your computation and database resources, background tasks are common strategies to do break down calculations for common payments needs such as reporting and analytics. When calculations are done in partial chunks, automated jobs that test whether those calculations have been done properly can reduce a lot of load for your support and developers when something goes wrong midway between a job, or failure.

  5. Negative/Failure Testing: Special card numbers provided by payments providers can help you recreate payment declines due to potential denial from processors for reasons such as not enough funds in account. You may also be able to test for rejections due to fraud and compliance. This helps lower the range of potential unknown errors your site may run into, especially when expanding to new markets or accepting more payment methods.

  6. Live testing: Live testing against payment providers is often tricky, and can led to accounts getting shut down if there is undue load on the API. Despite that, some testing in live is absolutely necessary before you can be confident that on release day, your integration is working as expected.

  7. Test for absence of sentive information: Storing user information such as credit card number or passwords is a very common way of violation of PCI compliance. Regex patterns can be used to make sure that neither your logs nor your database is storing sensitive information.

I intend to write more posts in this series, covering topics such as considerations before and after going live with payments, when scaling up and so on. If you liked this post, please share or comment.

If you have feedback on this blog post or integrating payments, please feel free to reach out!.

Takeaways From MicroConf Starter 2017

With the rise in VC funded startups, there was not a big community for individuals and small teams launching and supporting digital product businesses with their own profits. Rob Walling and Mike Taber noticed that need and created MicroConf, a conference for self-funded software startups.

I was at MicroConf Starter last week. In its 12th year, MicroConf split into Starter and Growth Tiers, the starter edition for people who do not make a full time income from digital products. If you are the kind of person who enjoys taking an idea to a functional product that solves real world problems, MicroConf is a conference where everyone has that shared goal.

Some at MicroConf have launched products and were doing quite well from a digital product business, let it be online courses, software plugins, SAAS etc. There were also number of people who wanted to learn more about find the right idea, product-market fit, sales and marketing.

I really enjoyed the pragmatic voice of the conference, keeping focus on balance. The conference does not shy away from the fact that it is not a easy task to bootstrap software products.

MicroConf has great notes for the whole conference. Instead of trying to go through the whole conference, here are some of my takeaways from MicroConf Starter 2017.

  1. Consistency: Rob Walling emphasized start of the conference that the success of MicroConf will be what these two days can do for the remaining 363 days of the year. Often consistency made the difference in the eventual endurance and success of the product. Josh Doddy’s blog was fairly dormant for the first 12-14 months but peaking exponentially near its current runtime of 18 months. Mastermind groups were mentioned as a great way for a group of people who help each other stay on track.

  2. Finding an idea worth building: Multiple speakers mentioned the need to take a hard look at your stocks and assets. What questions do people keep asking you? What are you passionate about that other people find boring? What would you from 6 months ago find valuable? All these were from Ben Orenstein’s talk, one of my favorite at the conference. Patrick Mckenzie also touched on the same topic, to double down on what you do very well already and what the market already buys from you. Justin Jackson mentioned the need to find the groups you are best equipped to serve, and to research the audience and find ideas rather than thinking in your own head what the problem could be. Mike Taber also emphasized focusing if you are in fact the right person to be building that product.

  3. When to launch: An MVP should solve a well defined problem, not solve a portion of it or solve every possible iteration of the problem. However, the lack of polish is intentional to see how much inconvenience the customers would endure to solve their problem. Justin Jackson had a great point that an MVP should be the smallest product you could build to disprove a hypothesis. It was interesting to see multiple speakers mentioning the importance of putting your face right by your product, to encourage trust and take responsibility of what you are delivering.

  4. On user acquisition: Probably the biggest concern of fledgling products, user acquisition/outreach had dedicated talks. Some of the key points where to focus on conversations with users and doubling down on a few approaches e.g. SEO, Content, Ads rather than throwing in the kitchen sink. Looking for integrations with other products by forming partnerships was a common theme. Key questions to ask users were to ask how they were solving the problem today, what they have tried or ask for introductions to someone with that problem. In the beginning, unscalable strategies such as concierge onboarding are useful, specially for SAAS products.

  5. Getting results: Users are the best signals here, and if you have to chase people down to use your products, it may not be solving a real problem. Google analytics charts showing growth and conversion rates were part of almost every presentation. At the same time, reading those charts can be a hard story, often showing charts recovering from a flatline or decline to eventual success because the founders believed enough to carry on. Its hard to think convictions to be infallible in the face of data however, and sometimes it is time to give up.

What Makes a Great Programming Tutorial?

The Internet started as a publishing medium. It excels at enabling people to share their unique gifts. An amazing amount of content gets put out on the web everyday, far beyond someone can read in a lifetime. Massive amount of information also means information overload.

In software, it is common to use web tutorials to supplement one’s learning of a particular material. Following guidelines of someone who has already done it can really fast track development. Tutorials appear in various forms in the web. Some of the common formats are long form blogposts, videos and series of email newsletters. Some enjoy the personal touch that videos can offer; others enjoy being able to quickly skim a blogpost.

When you want to put your hard earned knowledge and valuable time into writing a tutorial, there are questions worth thinking about. Does the tutorial cater to its intended audience? Will it reach them? Can someone quickly skim the content and still get value? Is there a way to measure the effectiveness of online tutorials?

Survey Questions

To look for answers, I asked a carefully selected group of 150 people about their preference of tutorials. The focus was on software engineers/designers/entrepreneurs due to my familiarity and experience with the field. What emerged from their responses gives a blueprint for creating great online resources.

  1. Real time feedback: If exercises or examples accompany a tutorial, multiple survey responders emphasized the need to check the user’s responses and provide interactive feedback to the user to guide them to the solution. Good examples are checkpoint multiple choice questions during Coursera/Udacity courses that you must complete to move forward with the course.

  2. Follow up: Several responders emphasized the need to provide contact information or other means to reach out to the author once they went through a tutorial. Providing a comments section or your email/twitter handle are great mediums for a reader to follow up.

  3. Address a concrete problem: Staying focused of a specific problem gives a tutorial depth. A differentiator can be to classify for the user whether the focus is academic and structured vs. a blogpost focused on solving a particular problem right now.

  4. Working examples: Interactive, working examples close to the content that builds on top of each other makes following along simpler. Michael Hartl’s Ruby on Rails tutorials came up as an example of detailed, comprehensive tutorials.

  5. Advanced user tutorials: Several user’s pointed out the need for tutorials that addresses advanced content. Making expectations clear about experience levels of intended audiences can be a huge positive for tutorials. Ray Wenderlich’s iOS tutorials are good examples of the detail and level of research and understanding that can happen before putting out content on the web.