Supercharge your Product Development with Infrastructure as Code

Why is there a need for IaC? What trouble will companies face if they don’t embrace IaC?

Without IaC, the IT infrastructure is configured, operated, and maintained manually. Historically, applications were developed using legacy IT infrastructure i.e. servers, storage, and compute. All these functions were provided using bare metal hardware in an on-prem environment. The configuration, operation, and maintenance were manually performed which meant high rates of human errors, delays, and demand for shedload members on the IT team. In addition, these manual processes exhibited lack of standardization of procedures and ineffective documentation, if at all. Collectively, this resulted not only in Environmental Drift but overall an inconsistent environment for application development and created even more challenges during scale-ups.

Explain to us the concept of IaC, and why should the companies work with it?

The irregularities we discussed in the absence of IaC, necessitate a more sophisticated solution for infrastructure to develop products. Infrastructure as Code (IaC) is that very revolution.

IaC, in contrast to the legacy manual model, is a more descriptive model which allows the configuration, provision, and management of infrastructure using machine-readable files.

With automated Infrastructure, configuration, operations, and maintenance functions are now performed using scripts. As a result, IaC gives you performance consistency, monitoring accountability, operations agility, scale-up flexibility,  audit transparency, software security, and a much faster development cycle. Companies that have embraced IaC benefit from reduced operations costs and rapid time to market new products, services, and features. All in all, higher revenue and more satisfied customers.

As a Start Up founder, what steps do I need to take to embrace IaC?

I would advise promptly embracing IaC or at least outlining the roadmap to immediately focus on embracing it.

The first step is to evaluate the infrastructure requirements for the products and/or services you offer. Secondly, with multiple options available, make a categorical decision on your tech stack according to your product.

Startups usually want to build and push products to the market rather quickly since they believe it’s merely prototype(s). However, I would encourage you to create a solid foundation with the right IaC principles and to implement IaC from the ground up. With a strong footing, scaling beyond the first 2- 3 servers will become streamlined and efficient with IaC.

Is IaC required to have DevOps in a Start Up? How are they related?

DevOps and IaC go hand in hand. One would not exist without the other. Although all companies apply the DevOps principles to varying degrees, the most popular is the Shift Left approach which is synonymous with the service ownership model. The idea is that the developers in IT teams are not working in silos but collaboratively, to create a holistic view of the entire application development lifecycle. In this spirit, developers are not only responsible for application development but also for creating the right infrastructure for the operations and deployment of the application as well. This means that the responsibility of coding has been fanned out among the IT team members. Testing and monitoring roles have been shifted left to the developers and all of this is enabled by IaC.

Do I need to test and monitor IaC?

There is no substitute for testing and monitoring IaC; it practices more stringent test requirements. Infrastructure can be automated flawlessly with the use of correct IaC templates and avoiding misconfigurations. An array of testing tools are available for you to choose but the fundamental notion of testing being critical, for IaC, cannot be overstated.

With QA functions disappearing in pure DevOps culture, who would carry out these stringent testing and monitoring?

In pure DevOps, CI/CD has automated IT testing and operations which in turn accelerates the process of application deployment exponentially. This results in continuous updates to your application and infrastructure. Now, if you don’t have automated testing, you are in a heap of trouble. At this rate of deployment, humans can not keep up and companies must implement automated testing strategies throughout the application development supply chain.

How would you address the IaC related fear of automation, in industry?

The idea is simple. If you anticipate failure, you can prepare for it and then mitigate it. The preparation you need is to implement a robust testing strategy and it is equally important to have a control feedback loop to continuously audit the strategy and improve. The ultimate goal is the provision of the right environment that doesn’t miss any build failures. Failures, caught and addressed in the next loop, will allow streamlined product deployment moving forward.

Explain the difference between declarative and imperative approaches for us.

The difference is that Declarative is a functional approach and Imperative is a procedural one. Declarative IaC approach answers the question ‘What is the preferred state’ and Imperative IaC approach answers the question ‘How is the preferred state achieved’’. Since it is critical to maintain your infrastructure in the desired state, it is recommended to use a declarative IaC approach. Imperative IaC approach relies on control loops and conditional statements which can become a complex endeavor.

Some supporting tools are available that create an imperative layer over the declarative model to provide an abstraction. Pulumi, for instance, is one such tool that is self declarative and can provide an imperative layer. Amazon CDK and Terraform are other examples of such tools that provide the best of both approaches.

Which of these approaches in your opinion can help the companies with their tech debt?

Traditional practices slow down the application development cycle and can lead to technical debts. In unpredictable cases e.g. immediate customer requirement, badly written code, or new feature request, the right automated testing strategies are your only way out of incurring technical debt. That is exactly what IaC promises. It creates those guard rails around your processes that reduce technical debt.

Mutable and Immutable infrastructures, which one would you recommend?

During application development, changes are inevitable. Your infrastructure will need to be scaled up or down, updated, and/or patches will have to be applied. Post-deployment, if the infrastructure can be changed, it is mutable infrastructure and vice versa. In case of immutable infrastructure, the changes are rolled out on a replica new machine before the old immutable infrastructure is taken down. There are multiple ways to go about it. However, when in doubt, go with mutable infrastructure.

In terms of tech stack, what are the best tools to implement IaC?

There is a spectrum of choices available, as discussed earlier. Amazon CDK, CDKFT, CloudFormation, Terraform, and Pulumi are all tools used to implement IaC. I suggest democratizing the decision among developers, SREs, and stakeholders since tech stack is not only meant for IaC but the entire application development pipeline is orchestrated. For developers, you have version control using Git; for operations CI/CD, AWS pipeline, GitOps; for code build service, there are custom Jenkins tools. Argo CD is a popular tool that operates specifically with Kubernetes-based deployments while Spinnaker allows you to deploy custom IaC technologies. However, the decision depends on your use cases and what is required to implement them.

My recommendation for IaC, is Terraform, as they offer one of the best tools, have an early mover advantage, a vibrant community, simplified syntax, and well-written documentation.

Does IaC help with the provisioning?

For Sure. IaC, much like software development, is version controlled in a centralized repository. Any update or change in features will be validated, tested, built, and pushed to required registries automatically. These automatic processes present a holistic development pipeline. All these tools come together to facilitate everything from integration to deployment and provisioning is definitely one aspect of the whole picture as well.

About IaC On Cloud Vs IaC On-Prem; your thoughts?

On-Prem infrastructures are heterogeneous when it comes to provisioning identity services, secret services, and/or network security. Limited automations can be performed on On-Prem environments. Most services have to be custom-built and are not scalable. On-Cloud, on the other hand, is standardized in service provision. Infrastructural resources become more flexible to scale up or down as required. Documentation best practices, abstractions at hand, repeatable and predictable environment are some of the factors that put IaC On-Cloud in a league of its own and hence offer more value.

What IaC profiles should I look for, as a StartUp founder?

Broadly speaking, for pure DevOps, Developers and Site Reliability Engineers (SRE) with the mindset of service ownership model are increasingly in-demand skills in Silicon Valley and across the world as well.

Tell us about the best practices for IaC in the industry?

To build an application infrastructure specifically in the agile world of IaC, the best practices require a set of foundational services at the periphery as well. For your company, best practices mean the acquisition of full package of foundational services i.e. Compute, storage, database, networking, security, all of it.

What does the future hold for IaC?

IaC has already saved a lot of money for different companies. It has sped up the process of software development from integration to deployment and ultimately delivers value to the customers. IaC helps create and serve those customers in a very fast, agile, and repeatable way.

In terms of the future, we already know manual infrastructure provision can not scale or perform at the same pace as IaC. Hence, lost customers and technical debt are natural outcomes. To enable those fast, repeatable deployments, and product iterations, IaC is the inevitable future and I very strongly believe that companies are and will implement full robust, orchestrated infrastructure, and pipelines at the heart of their businesses.

To Watch the Complete Episode, Please Click on This YouTube Link:

https://www.youtube.com/watch?v=d19JfKF5Y38&t=274s