The transition path to a microservices architecture is littered with the carnage of good intentions. Grant King, Senior Director of Engineering at 8x8, has been involved with five platform builds and describes it this way: “The first few projects paid for my education.”

When Dejan Deklich joined 8x8 as the new Chief Product Officer, he quickly realized 8x8’s transition to a microservices architecture was at a fork in the road, and wanted to make sure they chose the right path. He called Grant.

Having previously worked with Grant, Dejan laid out four clear objectives:

  1. Implement a platform for development, testing, building, and deploying microservices.
  2. Improve developer productivity.
  3. Abstract complexity, allowing engineers to focus on problem-solving instead of mechanics.
  4. Together with DevOps, provide a platform to managed microservices reliably.

With those objectives in mind and a wealth of experience to guide him, Grant quickly assembled a plan of action.

At its core was an 8-step program:

1. Educate and evangelize. Knowledge is knowing a tomato is a fruit. Wisdom is knowing tomatoes don’t go in fruit salad. With all of the enthusiasm about microservices, knowledge sometimes outpaces wisdom. It takes wisdom to know when to use them. A necessary first step is to educate and align engineering teams on all the new approaches and their oft-hidden complexities. For instance, when defining boundaries, engineers often make service scope too small and granular, having been told monoliths are bad. In reality, it's often easier to break a service into smaller pieces than join them later on. And the more services you have, the more complexity you'll introduce. Slow and steady wins the race as you build your microservices architecture.

2. New design thinking. Once there's a shared understanding and alignment on engineering approach, you have an opportunity to consider design thinking. One key areas often overlooked is complexity created by interaction between services. What happens if a service is not accessible? How does the design of each service ensure continued accessibility through updates? New ways of approaching design include using patterns such as:

  • Circuit breakers - what's the fallback logic if a dependent service goes down?
  • Eventual consistency - a single operation within a microservices environment can depend on multiple transactions. What happens if one of those transactions fails? You can't guarantee that the data is exactly as it should be but you can guarantee it will eventually get there.
  • Gracefully degraded behavior - if one part of the architecture goes down, degrade the user's experience so they can still operate with reduced functionality until the service comes back online.
  • Backward compatibility - API and schema changes must always be backward compatible. Without the privilege of schedule service downtime, changes need to be rolled into the environments seamlessly, so you can never coordinate breaking changes with services that depend on your contract.

3. Create an ecosystem. The ecosystem should include these components:

  • Container orchestration engine
  • Cluster management tooling
  • Monitoring and alerting infrastructure
  • Configuration management
  • Security
  • API Gateway with throttling
  • CI and CD
  • Standard libraries and development platform

There is growing interaction between these ecosystem components. A key to creating an effective ecosystem is facilitating integration with minimal overlap in functionality.

4. Focus on engineering productivity. The goal here is to empower engineering productivity without chaos. What ecosystem will maximize developer time spent...developing?

5. Own the tools engineers use. Surprisingly, this is frequently overlooked. It's critical to make all of the essential development and test applications necessary for the engineers to complete all steps of the development process.

6. Leverage existing processes. Let's be honest: have you every had the luxury of inventing new processes? We're almost always laying down tracks while the train is running. Use existing processes and modify them as required to drive an “automate while remaining compliant” approach. Lean on existing processes to quickly highlight issues or where new approaches are required.

7. Certification to maintain consistency. Now that everyone is on board, creativity abounds, and it's important to harness that creativity with minimal disruption. Implement a certification process to establish standards keeps everyone trained, knowledgeable, and wise on how best to work within the ecosystem. Here are some warning signs developers have gone off-track:

  • Too much manual scaffolding to get started
    • New services are a copy-paste of existing ones (proliferate different patterns and versions)
    • No restriction on usage of spring libraries and features. Define a set of standard patterns for leveraging libraries. Don't reinvent the wheel, especially when it comes to parts of the service not central to the business problem being solved.

Here’s the framework used to establish the initial certification program:

An engineering master class: Building a supportable, scalable and flexible microservice

An engineering master class in GDS:

  • Deploying into k8s
  • Debugging issues
  • Manage your monitoring / alerting / pager duties / minimize outages

Service certification: Services must meet defined standards to be deployed into production

8. Move quickly. One of the lessons from recent events is that change can happen quickly. Clear goals with achievable but aggressive timelines will help accelerate adoption. It sounds like a textbook, but it works - be prepared for failure, learn from the failure and adjust accordingly. Don't let paralysis by analysis get in the way of delivering.

“This type of transition requires a thick skin. As a new strategy and development model are imposed on the team, that initial adjustment includes some tension. The resistance isn’t personal, but it sure feels that way some days. It takes some perseverance and finesse to help the teams move from tension to adoption.” -Grant King

In eight months, the teams were able to achieve the initial goals and put the following ecosystem in place.

The 8x8 microservices architecture guiding principles:📷

  • Reduce administration time: Unified management, reporting, compliance, integrations, technical support
  • Accelerated innovation: Develop once, deploy many times
  • Increase engineering efficiency: Deliver cross-market use cases, development flexibility and efficiency
  • Remove risk: Clear accountability, fewer connection point vulnerabilities
Microservices_Guiding_Principles.png

Environment:

  • Global - running in 168 countries
  • Cloud agnostic
  • Full support for Kubernetes
  • Highly available active-active out-of-the-box
  • All deployments into production are fully auditable
  • Standards-based API integration framework
  • Auto scaling - provisioning scales dynamically
  • Real-time, 24x7 observability, alerting and escalation process to the NOC level
  • Two levels of automatic and configurable failover - service and network
  • Proven - based on Netflix open source components and Spring Boot framework
  • Optimizes development cycles - full service up, running and deployed in minutes - everything developers need is included
  • One platform for security and monitoring
  • Delivers consistent application of best practices so engineers can stay focused on development
Cloud8_Global_Deployment_Service.png

The impact of the transition so far.

  • 34 K8s clusters
  • 179 service teams
  • 399 services
  • 2000 deployments per month
  • 1,000 - 3,000 requests per second

The takeaways

A few key points to keep in mind when transitioning to a microservices architecture based on the learning from the 8x8 transition and previous transitions:

  1. Put guardrails around the flexibility of the microservices architecture to maintain consistency and ensure integration
  2. Optimize for developer efficiency by creating an ecosystem that has all of the tools and leverages, as much as possible, existing processes
  3. At the beginning, invest time through lunch and learns, team meetings, and 1:1 discussions to educate everyone on the tricky parts of a microservices architecture

We're proud to have Grant King at 8x8 and proud to feature him as an 8x8 Original Thinker. As we roll out more of these stories over the coming weeks, we encourage you to follow along. It's people like you, our customers, partners, and employees, that are helping all of us communicate, collaborate, and work better in the work-from-anywhere world. Thank you and see you next week!