Sunday, 5 July 2026

API-First Banking: Building Modern Digital Banks with Container Platforms, Enterprise API Gateways, and API Monetization

Introduction:

The world of banking industry is experiencing one of its largest technology transformations since the introduction of Core Banking Systems (CBS).Traditional banks once viewed APIs merely as integration mechanisms between applications. Today, APIs have evolved into business products capable of generating direct revenue, enabling digital ecosystems, accelerating innovation, and creating entirely new banking business models.

The combination of:

API-First Architecture
Open Banking
Banking-as-a-Service (BaaS)
Embedded Finance
Container Platforms (Red Hat OpenShift/K8S)
Enterprise API Gateways
Event-Driven Architecture

has become the digital foundation of modern day financial institutions.

Also the industry studies consistently show that leading banks like JPMorgan, Bank Of America, Wells Fargo, HSBC, SBI, HDFC and many more increasingly treat APIs as strategic products rather than technical assets, using them to accelerate digital transformation, improve customer experience, and create new revenue streams for the bank as the CASA is diving deep and forecasted to deep dive as well.

Why Banks are Moving to API-First

Traditional Banking Architecture:

Historically banking applications architecture were tightly coupled.

Internet Banking --> Middleware --> Core Banking --> Database

Problems included:

Slow product delivery
Difficult integration
Vendor dependency
ROI vs High maintenance cost
Monolithic deployments
Multiple point-to-point integrations

Launching a new banking product often required months of development & approvals. In particular to PSU banks the Choas were little deeper though governace and regulatory practicies as well.

What is API-First Banking?

Simple to the core as "API-First Banking means every banking capability is designed as an API before applications are built". ( But, not an template driven approach !!!)

Instead of asking:

"How should we build the application?"

Banks now ask:

"What banking service should this API expose?"

Examples include:

Customer Onboarding API
Account Opening API
Payment API
Loan Eligibility API
KYC API
Credit Score API
Balance Inquiry API
Transaction History API
Card Management API
Investment API

Applications become consumers of APIs rather than owners of business logic. Great isn't it !!!

API-First Design Philosophy:

The below flow will help understand us better:

Business Capability --> API Design --> OpenAPI Specification --> Mock API --> Development -->     ▼
Testing --> Deployment --> API Product

This approach enables parallel development, faster partner onboarding, standardized contracts, and iterative improvement.

Enterprise Container Architecture

Modern banks rarely deploy APIs directly onto virtual machines.

Instead they use enterprise Kubernetes orchestration platforms such as Red Hat OpenShift.

                   Internet
                     |
                     V
             Global Load Balancer
                     │
                     V
          Enterprise API Gateway
                     │
     ┌───────────────┼───────────────┐
     V               V               V
Authentication   Rate Limit     Threat Protection
     │
     V-------------------------------
Red Hat OpenShift Cluster
-------------------------------

Customer Service Pods

Account Service Pods

Loan Service Pods

Payment Service Pods

Notification Pods

Fraud Detection Pods

Kafka Event Streams

Redis Cache

Service Mesh

Monitoring Stack

Logging Stack

Secrets Management

Persistent Storage

-------------------------------
Backend Systems
-------------------------------

Core Banking

Cards

CRM

Treasury

SWIFT

Payment Switch

Mainframe

Data Warehouse

This architecture provides:

Horizontal scalability
High availability
Disaster recovery
Rolling upgrades
Auto healing
Zero downtime deployment

Why Containers Matter in Banking

Traditional VM Deployment

One VM
One Application

Problems

Resource wastage
Slow provisioning
Difficult scaling

Container Platform

Node

Customer API

Payment API

Loan API

Card API

Notification API

Benefits

Faster deployment
Lower infrastructure cost
Better resource utilization
Elastic scaling
Cloud portability

Enterprise API Gateway Architecture

The below flow will help to understand, how mordern bank's API Gateway is the digital front door of a bank.

Without API Gateway:

Consumer --> Core Banking

With API Gateway:

Consumer --> API Gateway --> Authentication --> Authorization --> Throttling --> Logging -->
Transformation --> Monitoring--> Backend Services

Enterprise Gateway Responsibilities

Authentication

OAuth2
OpenID Connect
JWT
Mutual TLS
SAML

Authorization

RBAC
ABAC
Consent validation

Traffic Control

Rate limiting
Spike arrest
Quotas
Request validation

Security

WAF
SQL Injection prevention
Bot protection
DDoS mitigation
Certificate validation

Protocol Transformation

REST ↔ SOAP

REST ↔ MQ

REST ↔ ISO 8583

REST ↔ ISO 20022

REST ↔ Kafka

Monitoring

Response time
Success rate
API usage
Latency
SLA monitoring

Popular Enterprise API Gateways

Gateway	Strength	Typical BFSI Use Case
IBM DataPower	Financial security, XML/JSON transformation, hardware acceleration.	Payment gateways, SWIFT, PCI-DSS
IBM API Connect	Full API lifecycle, developer portal, monetization	Open Banking, Partner APIs
MuleSoft	API-led connectivity, reusable integration assets	Digital banking transformation
Kong Gateway	Cloud-native, lightweight, Kubernetes integration	Modern microservices
Apigee	Enterprise analytics and developer ecosystem	FinTech partnerships
Red Hat 3scale	OpenShift-native API management	Hybrid cloud API governance

Event-Driven Banking

Not every banking operation should be synchronous.

Instead of:

Customer --> Payment API --> Notification API --> Fraud API --> Analytics API

Modern banks publish events.

Payment Completed -->Kafka-->Fraud Engine --> SMS Service --> Analytics-->CRM--> Data Lake

Benefits

Loose coupling
Better scalability
Faster processing
Real-time analytics

Enterprise Security Best Practices

Modern banking APIs should implement:

OAuth 2.0
OpenID Connect
Financial-grade API (FAPI)
Mutual TLS
JWT validation
End-to-end TLS encryption
Token introspection
API versioning
Zero Trust networking
Centralized secrets management
Continuous vulnerability scanning
Audit logging
PCI DSS controls
Data masking
Encryption at rest

Financial-grade API profiles build on OAuth 2.0 and OpenID Connect with stricter security requirements for high-value financial transactions.

API Lifecycle in Banking

Design

|
v

Review

|
v
Mock

|
v

Develop

|
v

Test

|
v

Security Scan

|
v

Deploy

|
v

Publish

|
v

Monitor

|
v

Version

|
v

Retire

Every API should be treated as a product with ownership, documentation, SLAs, analytics, and governance.

How BFSI Monetizes APIs

This is where APIs become profit centers.

1. Banking-as-a-Service (BaaS)

The bank exposes:

Account creation
Cards
Payments
KYC
Wallet
Lending

Third parties build applications using the bank's infrastructure.

Revenue

Monthly subscription
Revenue sharing
Transaction fee

2. API Marketplace

The bank publishes APIs.

Example

Developer Portal --> Browse APIs --> Subscribe --> Generate Key--> Pay --> Consume

Revenue

Per API call
Monthly plans
Premium support

3. Premium APIs

Free

Balance

Paid

Credit scoring
AML screening
FX rates
Investment data

4. Embedded Finance

Retailers --> Bank APIs --> Instant Loans --> Insurance --> Payments --> BNPL

The bank earns interchange, referral, lending, and service fees without customers visiting a bank branch.

Typical API Pricing Model

Tier	Monthly Fee	API Calls
Free	$0	10,000
Bronze	$199	100,000
Silver	$999	1 Million
Gold	$4,999	10 Million
Enterprise	Custom	Unlimited

Additional pricing may include:

KYC verification fee
Payment initiation fee
FX conversion fee
Fraud detection fee
Premium analytics fee

Real Banking API Products

Examples include:

Account Opening API
Payments API
Virtual Account API
QR Payment API
Card Issuance API
Loan Origination API
AML Screening API
Customer Identity API
Credit Bureau API
Wealth Management API
Treasury API

Each API becomes a measurable business asset.

Best Practices Checklist

✔ Design APIs before coding

✔ Follow OpenAPI specifications

✔ Use container platforms

✔ Adopt GitOps and CI/CD ( DevSecOps as your organization culture )

✔ Secure APIs with OAuth 2.0, mTLS, and FAPI

✔ Implement Zero Trust principles

✔ Use an enterprise API gateway

✔ Version APIs

✔ Maintain backward compatibility

✔ Publish developer documentation

✔ Provide sandbox environments

✔ Monitor API health continuously

✔ Automate security testing

✔ Track API adoption and revenue

Use Cases :

Digital customer onboarding :

Business Challenge:

Traditional account opening required customers to visit a branch with physical documents, leading to lengthy turnaround times and high operational costs.

+---------------------------------------------------+

| Mobile Banking App |

+---------------------------------------------------+

| Enterprise API Gateway |

|---------------------------------------------------|

| - Authentication (OAuth2 / JWT / mTLS) |

| - Authorization |

| - Rate Limiting |

| - API Security |

| - Request Validation |

| - Logging & Monitoring |

+---------------------------------------------------+

| Customer Onboarding API |

+---------------------------------------------------+

+---------------+---------------+

| | |

v v v

+-------------+ +-------------+ +----------------+

+-------------+ +-------------+ +----------------+

| | |

+---------------+---------------+

+-------------------+

| Aadhaar |

| Verification API |

+-------------------+

| AML Screening |

+-------------------+

+---------------------------------------------------+

| Core Banking System |

+---------------------------------------------------+

| Customer Account Created Successfully |

+---------------------------------------------------+

Flow Explanation

Mobile Banking App sends a request to open a new account.
The Enterprise API Gateway authenticates the user, authorizes the request, enforces security policies, and forwards the request.
The Customer Onboarding API orchestrates the complete onboarding workflow.
Identity Verification API validates the customer's identity.
eKYC Service performs electronic Know Your Customer verification.
PAN Verification validates the customer's PAN details with the authorized service.
Aadhaar Verification validates Aadhaar information (where applicable and permitted by regulation).
AML Screening checks the customer against anti-money laundering watchlists and compliance rules.
If all validations succeed, the Core Banking System creates the customer profile and account.
A successful response is returned to the mobile application, allowing the customer to begin using banking services.

_________________________________________________________________________________

Conclusion:

API-First Banking is no longer optional—it is the architectural foundation of modern financial services. Institutions that successfully combine container platforms, enterprise API gateways, event-driven integration, and disciplined API governance can innovate faster, integrate partners more efficiently, and unlock new revenue through Banking-as-a-Service, Open Banking, and Embedded Finance.

The future belongs to banks that treat APIs as products, containers as the standard runtime, and platforms as the engine of continuous innovation. By investing in secure API management, cloud-native architecture, and monetization strategies, BFSI organizations can transform technology investments into measurable business growth while delivering resilient, scalable, and customer-centric digital banking experiences.

Why : ESB is out, APIs are in ? : The Future of Enterprise Banking Architecture (ESB, Middleware, and the API-First )

Introduction: The Role of ESB and Middleware in Modern Banking

As I see for the last two decades, the Enterprise Service Bus (ESB) has sat at the heart of most of the BFSI IT as a central nervous system that routes messages between core banking systems, payment gateways, CRM platforms, risk engines, and countless third-party services. Middleware, broadly speaking, is the connective tissue that lets these disparate systems often built decades apart, on different technologies, by different vendors which talk to each other without every application needing to know the internal details of every other one.

In banking specifically, this layer has always mattered more than in most other industries. A single bank's retail transaction might touch a core banking system (CBS), a fraud detection engine (e-FRM), a ledger, a notification service (SMS), and a regulatory reporting pipeline should complete all within milliseconds. The ESB traditionally handled this through message transformation, protocol mediation (SOAP to MQ to REST, for instance), orchestration, and centralized routing logic.

But the ESB model, unfortuantely while still present in many banks, is increasingly seen as a bottleneck rather than an enabler. Centralized orchestration creates a single point of failure and a governance chokepoint where every new integration needs to go through the same overloaded team, the same brittle canonical data models, the same monolithic deployment cycles, etc. As banks race to support open banking, embedded finance, and real-time payments, this centralized middleware pattern struggles to keep pace.

Why APIs Are Becoming the New Architectural Backbone

The shift underway in enterprise banking architecture is a move from centralized orchestration (ESB) to decentralized, API-led connectivity. Instead of a bus that owns all integration logic, banks are building layered API ecosystems:

System APIs here expose core banking, ledger, and legacy mainframe capabilities as clean, reusable services, hiding the complexity of the underlying systems.
Process APIs here orchestrate multiple system APIs into meaningful business capabilities, such as "initiate a loan application" or "execute a cross-border payment."
Experience APIs here tailor those business capabilities to specific channels: mobile apps, partner portals, third-party fintech integrations, or internal dashboards.

This layered approach which is often associated with API-led connectivity frameworks decouples change. A mobile team can iterate on an experience API without waiting on the core banking team. A new fintech partnership can be exposed through a process API without touching the ledger system at all.

API templates accelerate this further. Rather than every team designing an API from scratch, banks are standardizing templates for common patterns likely as for a payment initiation, account information, KYC verification, consent management with pre-approved data models, security controls, versioning conventions, and documentation structures. This is particularly relevant in markets shaped by open banking regulation, where APIs must conform to standards like the UK's Open Banking specification, the EU's PSD2/Berlin Group framework, or India's Account Aggregator and UPI ecosystems. Templates turn regulatory compliance from a bespoke project into a repeatable, governed pattern.

The direction of travel is clear: middleware is not disappearing, but it is being reshaped. ESBs are giving way to API gateways, service meshes, and event-driven backbones (Kafka, event streaming platforms) that support asynchronous, real-time, and high-throughput banking use cases like an instant payments, fraud scoring, real-time account alerts whcih will far better than traditional request-response ESB patterns ever could.

Architecture Considerations for Open APIs in Enterprise Banking

Designing Open APIs for a bank is a fundamentally different exercise than designing internal APIs, because the consumers are external, often unknown in advance, and operating under regulatory scrutiny. Key considerations include:

1. API Gateway as the enforcement point The gateway becomes the control tower for handling authentication, rate limiting, request validation, routing, and analytics. In banking, this is also where regulatory obligations (like strong customer authentication) get enforced consistently, rather than relying on every backend team to implement them correctly.

2. Standardized specifications and versioning Open API (Swagger/OpenAPI) specifications should be treated as contracts, not documentation afterthoughts. A robust versioning strategy with semantic versioning, deprecation policies, backward compatibility windows whcih is essential because banks cannot break external fintech integrations without significant reputational and regulatory consequences.

3. Consent and data minimization For account information and payment initiation APIs, architecture must bake in explicit customer consent management, scope-limited access tokens, and data minimization whcih will exposing only what's needed for a given use case, not the full customer record.

4. Idempotency and reliability Payment and transaction APIs must be designed with idempotency keys, retry-safe operations, and clear failure semantics, since network retries in distributed banking systems can otherwise cause duplicate transactions.

5. Sandbox and developer experience A mature Open API program includes a sandbox environment, comprehensive documentation, and a developer portal; treating third-party developers and fintechs as first-class users of the bank's platform, not an afterthought.

6. Interoperability with legacy core systems Realistically, most banks still run core systems from the 1990s or 2000s. The API layer must act as an anti-corruption layer for translating modern REST/JSON semantics into the batch, mainframe, or proprietary protocols underneath, without leaking that complexity to API consumers.

Securing APIs Against AI-Driven Threats

The security conversation around banking APIs has changed considerably as AI has become both a defensive tool and an attack vector. A few architectural considerations stand out:

AI-powered attack patterns to defend against:

Automated credential stuffing and account takeover at machine speed, using AI to adapt to rate-limiting and CAPTCHA defenses in real time.
AI-generated synthetic identities used to pass KYC checks or open fraudulent accounts through onboarding APIs.
Prompt injection and manipulation of AI-driven customer service or agentic banking assistants that have API access to account actions with a growing risk as banks deploy LLM-based agents with transactional capabilities.
API scraping and reconnaissance using AI to rapidly probe API surfaces for undocumented endpoints, weak authorization checks, or business logic flaws (like broken object-level authorization, currently one of the most common API vulnerabilities in banking).

Architectural defenses:

Zero-trust API access: Each with every request authenticated and authorized regardless of network origin, with mutual TLS between internal services and OAuth 2.0/OIDC with short-lived tokens for external access.
Fine-grained, attribute-based authorization: here by checking not just "is this user authenticated" but "is this specific user authorized for this specific account/resource," which directly addresses broken object-level authorization risks.
AI-driven anomaly detection at the gateway layer: Here by using behavioral baselines to flag unusual API call patterns (volume, sequence, timing) that suggest automated abuse, effectively fighting AI-driven threats with AI-driven defense.
Guardrails around agentic AI integrations: if a bank exposes APIs to AI agents (internal copilots or customer-facing assistants), those integrations need scoped permissions, human-in-the-loop approval for high-risk actions (large transfers, limit changes), and strict validation of any AI-generated API calls before execution.
Continuous API discovery and posture management: Here many breaches happen through "shadow APIs" that security teams don't even know exist; automated API inventory and posture scanning should be a standing architectural function, not a one-time audit.
Threat modeling specific to LLM-integrated APIs — treating prompt injection, data exfiltration through AI responses, and model manipulation as first-class threats in the API threat model, not an afterthought bolted onto traditional API security.

How Gateway Platforms Like IBM Cloud Pak for Integration, Apigee, Kong, and MuleSoft Fit In

None of the architecture above is realized purely on paper but it's implemented through API management and integration platforms that provide the gateway, developer portal, security enforcement, and lifecycle governance as a product. Each of the major platforms takes a slightly different angle, which matters when a bank is choosing (or already running) one.

IBM Cloud Pak for Integration (CP4I) CP4I is aimed squarely at large, legacy-heavy enterprises like banks that still run significant mainframe and WebSphere-based ESB infrastructure. It bundles API management (built on IBM API Connect), an integration/ESB engine (App Connect), event streaming (Kafka-based), and MQ messaging into one containerized, Kubernetes-native platform. Its architectural value in banking is bridging old and new: it lets a bank keep its existing IBM MQ and mainframe investments while gradually exposing them as governed APIs, rather than forcing a rip-and-replace. It's a strong fit where the bank's core is IBM-centric, enterprise is heavily as an IBM SHOP and modernization needs to be incremental.

Google Apigee Apigee positions itself primarily as a full-lifecycle API management platform rather than an integration/ESB replacement, it assumes the backend services already exist (or are built elsewhere) and focuses on exposing, securing, monetizing, and analyzing APIs. Its strengths for banking architecture are strong developer portal tooling (important for Open Banking/fintech ecosystems), fine-grained traffic policies, quota and monetization models for partner APIs, and deep analytics on API consumption patterns which will be useful for spotting anomalous usage that might indicate fraud or abuse. Apigee is often chosen when the bank's priority is the external-facing Open API and partner ecosystem layer specifically.

Kong Kong is a lightweight, high-performance API gateway built on NGINX, popular in cloud-native and microservices-heavy environments. Its architectural appeal is its plugin-based extensibility (authentication, rate limiting, logging, transformation) and strong fit with Kubernetes and service mesh patterns (Kong Mesh). For banks pursuing a decentralized, microservices-first architecture rather than a heavy centralized platform, Kong offers a leaner alternative and often deployed per-domain or per-team rather than as one enterprise-wide gateway, which aligns with the API-led connectivity model discussed earlier. It's frequently the choice for digital-native banking subsidiaries or neobanks that don't carry decades of legacy integration debt.

MuleSoft (Anypoint Platform) MuleSoft is built explicitly around the System/Process/Experience API layering model may be it's arguably the platform most associated with popularizing that pattern. Anypoint Platform provides design, build, and management tooling across all three layers, plus strong pre-built connectors to enterprise systems (SAP, Salesforce, mainframes, databases), which shortens the path from legacy system to reusable System API. For banks in the middle of an ESB-to-API transition, MuleSoft is often chosen precisely because it offers a structured methodology, not just gateway infrastructure and it guides teams toward the layered architecture rather than leaving them to design it themselves.

Architectural takeaway There isn't a single "correct" platform to choose and the right choice depends on where a bank currently sits on the ESB-to-API journey:

Heavy IBM/mainframe legacy, Enterprise armed with IBM SHOP, incremental modernization : CP4I
External Open Banking/partner ecosystem as the priority: Apigee
Cloud-native, microservices/service-mesh architecture: Kong
Structured System/Process/Experience transformation with strong legacy connectors: MuleSoft

Many large banks like NatWest, WellsFargo, HDFC, SBI has architecture practice to run more than one of these tools for example, MuleSoft or CP4I for internal integration and System/Process APIs, with Apigee or Kong at the edge for external-facing, high-traffic Open Banking APIs. The architectural principle that matters more than the vendor choice is keeping the gateway as a consistent policy enforcement point (authN/authZ, rate limiting, threat detection) regardless of how many platforms are in play, so security and governance don't fragment across tools.

Conclusion

The banking middleware layer is undergoing its most significant transformation since the ESB era began. The future is not "ESB vs. APIs" but a layered, API-first architecture where the ESB's remaining relevant functions along with the protocol mediation, legacy integration whcih are absorbed into a broader fabric of API gateways, event streaming platforms, and well-governed API templates. Open banking has made external API exposure a regulatory and competitive necessity, and the rise of AI era for both as an attacker's tool and a defender's ally which realy means API security architecture can no longer be an afterthought bolted onto the gateway. Banks that treat their API layer as a strategic platform, governed with the same rigor as their core banking systems, will be the ones positioned to move fast on open banking, embedded finance, and AI-native banking experiences without compromising the trust their entire business is built on.

Please comment your thoughts and share your enterprise API adoption. Next blog will be focused on the technical architectures

Saturday, 20 June 2026

ETCD in Red Hat OpenShift: Why Heartbeat Monitoring Matters and How VMFS Storage Can Make or Break Your Cluster

Introduction

When discussing Red Hat OpenShift performance and reliability, most administrators focus on worker nodes, application pods, networking, and ingress traffic. However, the true foundation of every OpenShift cluster is ETCD.

ETCD is the distributed key-value store that acts as the source of truth for Kubernetes and OpenShift. Every object in the cluster-including deployments, pods, services, routes, secrets, config maps, and node information-is stored in ETCD.

Because ETCD sits at the heart of the OpenShift control plane, even minor performance degradation can impact the entire cluster. One of the earliest indicators of ETCD health is heartbeat latency. In many VMware-based OpenShift deployments, ETCD heartbeat issues are not caused by Kubernetes itself but by underlying storage latency, particularly within VMFS-backed datastores.

This article explains:

The role of ETCD in OpenShift
Why heartbeat monitoring is critical
How storage latency affects ETCD
The relationship between ETCD and VMware VMFS storage
A practical script to monitor ETCD latency and heartbeat health

Understanding ETCD in OpenShift

ETCD is a distributed, strongly consistent key-value database used by Kubernetes and OpenShift.

Every operation performed through the Kubernetes API eventually results in ETCD activity.

Examples include:

Creating a pod
Scaling a deployment
Updating a secret
Modifying a route
Registering a node
Updating cluster configuration

In a standard OpenShift deployment, ETCD runs on the control plane nodes and uses the Raft consensus algorithm to maintain consistency across multiple members.

The architecture ensures that:

Data is replicated across ETCD members
Changes are committed only after consensus
Cluster state remains consistent even during failures

Without a healthy ETCD cluster, OpenShift cannot function reliably.

The Importance of ETCD Heartbeats

ETCD relies on the Raft consensus protocol.

Within Raft, a leader node periodically sends heartbeat messages to follower nodes.

These heartbeats serve several purposes:

Confirm leader availability
Maintain cluster synchronization
Prevent unnecessary elections
Ensure consistency among ETCD members

A healthy ETCD cluster continuously exchanges heartbeats with minimal latency.

If followers stop receiving heartbeats within the expected timeframe, they assume the leader has failed and initiate a leader election.

While occasional elections are normal, frequent elections indicate underlying infrastructure problems.

Why ETCD Heartbeat Monitoring Is Essential

1. Detects Infrastructure Problems Early

Heartbeat delays are often the first warning sign of:

Storage latency
Network congestion
CPU starvation
Memory pressure

Administrators who monitor ETCD heartbeat behavior can identify issues before applications begin failing.

2. Prevents Control Plane Instability

Delayed heartbeats can trigger unnecessary leader elections.

This can result in:

Kubernetes API delays
OpenShift console slowness
Failed deployments
Operator degradation
Cluster upgrade interruptions

3. Improves Troubleshooting

Many OpenShift issues initially appear as:

API server slowness
Cluster operator degradation
Node instability

ETCD heartbeat metrics help quickly identify whether the root cause lies in the control plane infrastructure.

ETCD and Storage: A Critical Relationship

Unlike traditional databases that often depend on throughput, ETCD depends heavily on low latency.

Every ETCD transaction involves:

Writing data to disk
Performing fsync operations
Replicating changes to followers
Receiving acknowledgements

Because these operations occur constantly, storage latency directly affects ETCD responsiveness.

A few milliseconds of additional latency may seem insignificant, but in ETCD environments those delays accumulate rapidly and impact heartbeat processing.

Understanding VMware VMFS Storage

Many enterprise OpenShift clusters run on VMware vSphere.

In these environments, virtual machines often reside on VMFS (Virtual Machine File System) datastores.

VMFS provides:

Shared storage access
High availability
vMotion support
Centralized datastore management

While VMFS is highly reliable, ETCD performance ultimately depends on the performance characteristics of the storage array beneath VMFS.

How VMFS Storage Affects ETCD Heartbeats

Storage Latency

The most common cause of ETCD performance degradation is storage latency.

When datastore latency increases:

WAL writes become slower
Backend commits take longer
Raft operations are delayed
Heartbeats arrive late

Administrators may begin seeing warnings such as:

"leader failed to send out heartbeat on time"

Datastore Contention

Many organizations place numerous virtual machines on the same datastore.

Examples include:

Database servers
Backup workloads
Application servers
OpenShift control plane nodes

When storage-intensive workloads compete for resources, ETCD often becomes one of the first services affected.

Queue Depth and Storage Congestion

Storage arrays have finite resources.

Issues such as:

Excessive queue depth
Controller bottlenecks
SAN congestion
Storage processor saturation

can increase datastore response times and negatively impact ETCD heartbeat processing.

Common Symptoms of Storage-Induced ETCD Problems

Administrators may observe:

Frequent ETCD leader elections
Slow API server responses
Delayed pod scheduling
Cluster operator degradation
OpenShift console timeouts
Upgrade failures
Node status update delays

In many cases, these symptoms originate from storage latency rather than ETCD software defects.

Key ETCD Metrics to Monitor

The following metrics are particularly important:

Metric	Purpose
etcd_disk_wal_fsync_duration_seconds	WAL write latency
etcd_disk_backend_commit_duration_seconds	Backend database commit latency
etcd_server_leader_changes_seen_total	Leader election frequency
etcd_network_peer_round_trip_time_seconds	Peer communication latency
etcd_server_has_leader	Leader availability

A sudden increase in these metrics should trigger investigation.

ETCD Health Monitoring Script

The following script can be executed from a system that has access to the OpenShift cluster and is authenticated using the oc command-line utility. (Please note : I will be sharing if requested in the comment only ) .

Interpreting the Results

Healthy Cluster

Typical characteristics:

WAL fsync latency below 10 ms
Backend commit latency below 25 ms
Minimal leader changes
No heartbeat warnings

Warning State

Typical indicators:

WAL fsync latency above 50 ms
Commit latency above 100 ms
Occasional leader changes
Sporadic heartbeat warnings

At this stage administrators should review datastore performance.

Critical State

Typical indicators:

Persistent latency above 100 ms
Frequent leader elections
API responsiveness issues
Control plane instability

Immediate investigation of storage infrastructure is recommended.

VMware Metrics to Correlate

Whenever ETCD latency rises, VMware administrators should review:

Datastore latency
Device latency
Kernel latency
Queue depth
Storage throughput
Storage controller utilization

Correlating OpenShift ETCD metrics with VMware performance statistics often leads directly to the root cause.

Best Practices

To maintain healthy ETCD performance:

Use Fast Storage

Prefer:

SSD-backed datastores
NVMe-backed storage
Low-latency SAN configurations

Isolate Control Plane Workloads

Avoid placing:

Backup servers
Large databases
High-I/O applications

on the same datastore as OpenShift control plane nodes.

Monitor Continuously

Track:

ETCD latency
Heartbeat behavior
Leader changes
VMware datastore performance

Investigate Heartbeat Warnings Immediately

Heartbeat warnings are often early indicators of larger infrastructure issues.

Ignoring them can eventually lead to cluster instability.

Conclusion

ETCD is the most critical component of the OpenShift control plane. While administrators often focus on CPU, memory, and networking, storage latency remains one of the most common causes of ETCD performance degradation.

Heartbeat monitoring provides an early-warning system that can reveal underlying infrastructure problems before they affect applications. In VMware environments, VMFS datastore performance directly influences ETCD write latency, commit operations, and Raft heartbeat processing.

By continuously monitoring ETCD heartbeat health, storage latency, and VMware datastore metrics together, organizations can significantly improve OpenShift stability, reduce unexpected outages, and ensure a resilient Kubernetes platform.

Note : This ETCD latency we experienced and solved in our on prem clusters, we welcome your way of experience and admin challenges related to ETCD.

Sunday, 14 June 2026

DevSecOps in 2026: The Trends Reshaping the Pipeline and the Tools Behind Them

A few years ago, "DevSecOps" mostly meant adding a security scanning step somewhere near the end of a CI/CD pipeline and calling it a day. That era is over. In 2026, security is being pulled earlier into development, automated more aggressively, and — increasingly — built to handle a development process where a meaningful share of code is written by AI rather than humans.

This post walks through where DevSecOps stands right now: the trends driving the biggest changes, the tools teams are actually using, and a few recent events that show exactly why this space needs constant attention.

The State of the Market

DevSecOps adoption has moved well past "early trend" status. A growing share of organizations now build software using DevSecOps practices, and the shift isn't just about checkbox compliance — security teams report real confidence gains, with automation increasingly seen as core to the approach rather than a bolt-on. Most teams now run some combination of static analysis (SAST), software composition analysis (SCA), container scanning, and infrastructure-as-code (IaC) checks as standard practice.

But adoption of tools isn't the same as maturity. The honest state of things in 2026 is that most teams have plenty of scanners running — the harder problem is what happens after a scanner finds something. If high-risk findings sit untriaged, more scanning just means more noise.

Trend 1: Consolidation and Alert Fatigue

The single biggest theme this year is consolidation. Teams that adopted a different point solution for each concern — one tool for SCA, another for SAST, another for container scanning, another for secrets — are now drowning in disconnected alerts from a dozen dashboards.

The response has taken two forms:

Unified platforms. Tools like Wiz and GitLab's built-in security suite aim to cover multiple domains (cloud posture, container security, SAST, DAST) from a single product, reducing the number of places a team has to look.
Aggregation layers. Open-source tools like DefectDojo sit on top of an existing mix of scanners and consolidate their output into one place — useful for teams that don't want to rip out tools that already work well individually, but need a single view of what actually matters.

A related shift is policy as code (PaC) — instead of manual compliance checklists, security and compliance rules are written as code (often using Open Policy Agent) and enforced automatically across pipelines and infrastructure. This turns "did we follow the policy?" from an audit question into something the pipeline can answer for itself.

Trend 2: Cloud-Native Security Goes Mainstream (CNAPP)

Cloud-Native Application Protection Platforms — CNAPPs — bundle together Cloud Security Posture Management (CSPM), Cloud Workload Protection (CWP), and IaC security into one platform that automatically discovers and protects new cloud resources as they're deployed.

This category got a lot more attention in 2026 after Google's roughly $32 billion acquisition of Wiz closed — one of the largest deals in the cloud security space, and a clear signal of how central cloud-native posture management has become to the broader security stack. For teams running multi-cloud or heavily containerized environments, CNAPP-style tooling is increasingly treated as table stakes rather than a "nice to have."

Trend 3: Supply Chain Security Gets Serious — and Gets Tested

Software supply chain security — SBOMs (Software Bills of Materials), artifact signing, and dependency provenance — has moved from "good idea" to "expected practice," with open-source tools like Syft and CycloneDX for SBOM generation and Sigstore for artifact signing becoming close to standard.

But 2026 also delivered a blunt reminder of why this matters: Trivy's own GitHub Action was compromised twice in the same month through tag poisoning — meaning a tool widely used to scan for supply chain risks briefly became a supply chain risk itself. It's a useful (if uncomfortable) case study: pinning action versions to commit hashes rather than mutable tags, and treating your security tooling's own dependencies with the same scrutiny as your application code, isn't paranoia — it's now a documented attack pattern.

Trend 4: AI Is Changing Both Sides of the Equation

AI shows up in DevSecOps in two very different ways right now, and it's worth keeping them separate:

AI as a target. AI coding assistants are generating code — and APIs — faster than security review can keep up, and a large share of security professionals now use or plan to use AI/ML for code review specifically because of this volume increase. At the same time, AI agents are increasingly consumers of internal APIs and systems, which creates a new category of identity and access management problem: how do you apply least-privilege and audit trails to a non-human actor making autonomous decisions?

AI as a tool. On the defensive side, AI-driven triage is one of the most-cited 2026 trends — using models to cut through alert noise, prioritize findings by actual exploitability rather than raw CVE severity, and in some cases draft remediation pull requests automatically. At a recent major security conference, "AI agent security" was reportedly the subject of products from a striking number of vendors in a single week — a sign of how quickly this sub-category is forming, even if much of it is still maturing.

The Tools Teams Are Actually Using

Rather than a generic "top 10" list, here's a more practical breakdown by what each tool is genuinely good at — because in 2026, "what's the best DevSecOps tool" is less useful than "what's the best tool for this specific job."

For container and dependency scanning (SCA): - Trivy (open source, by Aqua Security) remains the default choice for fast, free container image scanning, IaC misconfiguration checks, secrets scanning, and SBOM generation — all from a single CLI binary, with no feature gates. - Grype is a close open-source alternative focused specifically on container image CVE scanning, built on well-maintained vulnerability databases. - Snyk offers a more polished, dashboard-driven experience with automated fix pull requests and exploitability-based prioritization — useful when alert noise is the bigger problem than scan coverage.

For static analysis (SAST): - Semgrep has become a popular open-source option for fast, customizable static analysis across many languages. - Snyk Code offers a proprietary AI-assisted SAST engine with real-time IDE feedback, which Trivy notably does not provide at all.

For infrastructure as code: - Checkov and tfsec are widely used for Terraform and broader IaC policy scanning, often used together for breadth and speed respectively.

For runtime and cloud posture: - Wiz focuses on deep runtime visibility and IAM risk detection across cloud environments — particularly strong for cloud-heavy teams, and now backed by Google's resources following the acquisition. - Falco (a CNCF open-source project) handles runtime threat detection specifically for Kubernetes. - Sysdig is notable for forensics — capturing deep system call data so security teams can reconstruct what happened inside a container even after it's gone, which matters because containers are ephemeral by nature.

For secrets detection: - TruffleHog and GitGuardian both scan git history (not just current code) for leaked credentials and high-entropy strings — important because a secret committed and later removed is still exposed in history.

For pulling it all together: - DefectDojo remains the leading open-source option for aggregating findings across all of the above into a single, manageable view — arguably more important in 2026 than any individual scanner, given how fragmented the tool landscape has become.

A common pattern for teams building a stack from scratch: Trivy and Semgrep cover the widest surface area at zero licensing cost (containers, IaC, OSS CVEs, and SAST), with a commercial tool like Snyk layered in specifically for its developer workflow integration and automated remediation — rather than trying to replace the open-source layer entirely.

What This Means in Practice

If there's one thread connecting all of this, it's that tool count was never the goal — closing the loop between detection and remediation is. A team running ten scanners that never triages findings is, in a meaningful sense, less secure than a team running two scanners that consistently fixes what it finds.

For 2026 specifically, three things are worth prioritizing regardless of company size:

Pin your CI/CD dependencies — including your security tools' own GitHub Actions — to commit hashes, not tags. The Trivy incident this year is exactly why.
Don't treat AI-generated code differently in your pipeline — it should go through the same SAST/SCA/IaC gates as anything else, ideally with no exceptions for "the AI wrote it quickly."
Invest in triage and aggregation before adding more scanners. If your team already has signal it isn't acting on, a new tool will only add more signal you won't act on.

What's your stack looking like in 2026 — consolidated platform, mixed open-source, or somewhere in between? I'd be curious to hear what's working (and what isn't) in the comments.

API Discovery for enterpise

If you ask most engineering leaders "how many APIs does your organization actually have?", the honest answer is usually some version of "we're not entirely sure." That uncertainty used to be a minor governance headache. In 2026, it's a serious security and compliance problem — and it's the reason API discovery has become one of the fastest-growing categories in enterprise tooling.

This post breaks down what API discovery actually means, why it has become urgent now, and which tools are worth knowing about if you're evaluating this space.

What Is API Discovery?

At its simplest, API discovery is the process of automatically finding, cataloging, and documenting every API running across an organization's environment — internal services, partner integrations, third-party connections, and anything exposed publicly. This includes the APIs your teams know about, but more importantly, the ones they don't.

Three categories tend to come up constantly in this space:

Shadow APIs — endpoints that exist and are live, but were never registered in any official inventory
Zombie APIs — old or deprecated endpoints that were supposed to be retired but are still reachable
Orphaned APIs — services left behind after a team moved on, a project was shelved, or ownership changed without documentation following

Modern discovery tools find these by analyzing network traffic, inspecting API gateways, scanning code repositories and CI/CD pipelines, or some combination of all three.

Why This Has Become Urgent

API sprawl isn't new, but a few trends have made it dramatically worse recently.

AI coding assistants are generating APIs faster than teams can track them. What used to take a developer a day to scaffold can now be generated in minutes with tools like GitHub Copilot or other AI code generators. The speed is genuinely useful, but it widens the gap between "API exists" and "API is documented, secured, and governed."

AI agents are now API consumers too. It's no longer just human developers calling APIs — autonomous agents are increasingly calling them as part of automated workflows. Every one of those calls is a dependency that needs to be tracked, authenticated, and rate-limited appropriately, which multiplies the surface area that needs visibility.

Most enterprises run multiple API gateways, not one. It's common for one team to standardize on Kong, another on AWS API Gateway, and a third on Apigee or MuleSoft — each with its own catalog. The result isn't usually a discovery problem so much as a fragmentation problem: APIs get documented in one place, monitored in another, and governed nowhere consistently.

Put together, this means the old approach — a spreadsheet someone updates occasionally — simply can't keep up anymore.

The Current Tooling Landscape

The tools in this space generally fall into a few overlapping groups: security-focused discovery platforms, gateway-aggregation tools, and developer-collaboration networks. Here's a look at some of the names that come up most often.

Security-First Discovery Platforms

These tools approach API discovery primarily through the lens of attack surface management — finding what's exposed, flagging risk, and feeding that into broader security workflows.

Levo.ai takes a runtime-first approach, using lightweight sensors to observe API traffic at the OS layer before encryption. This lets it build a live inventory across REST, GraphQL, gRPC, and SOAP without requiring code changes, and it enriches each discovered endpoint with details like authentication method and sensitive data exposure.
Salt Security focuses on identifying APIs through traffic analysis and behavioral patterns, which is particularly useful for catching APIs that were never intentionally published anywhere.
StackHawk pairs discovery with API-focused dynamic application security testing (DAST), so once an API is found, it can be tested immediately rather than just logged for later review.
Traceable.ai, Akto, Qualys, Rapid7, Invicti, Akamai, and Orca Security all show up regularly in this category too, generally differentiated by how deeply they integrate with existing security stacks and cloud environments.

Gateway Aggregation and Inventory Tools

For organizations dealing with the "multiple gateways, multiple catalogs" problem specifically, some tools focus on pulling everything into a single unified view.

DigitalAPI.ai is built around this exact use case — aggregating APIs across Kong, Apigee, AWS API Gateway, Azure, MuleSoft, and others into one inventory, which helps when different teams have standardized on different platforms over time.

Developer Collaboration and Internal Discovery

Not every discovery problem is a security problem — sometimes it's simply "our own developers don't know this API already exists, so they build a duplicate."

Postman's Private API Network addresses this directly. Available on Postman's enterprise tier, it gives internal developers a searchable network of workspaces, collections, and documented APIs built by other teams — reducing duplicate work and onboarding time. Postman frames this as a "git-driven workflow," where the catalog stays in sync with what's actually in source control rather than relying on someone remembering to update a wiki.
The same company's Public API Network is worth knowing about too, even if it's a different use case — it's reportedly one of the largest public directories of APIs, with well over 100,000 listed, complete with ready-to-fork collections and documentation.

How to Think About Choosing One

If you're at the stage of evaluating these tools, a few questions tend to matter more than feature checklists:

Internal or external focus (or both)? Security platforms tend to emphasize finding what's exposed to the outside world; collaboration tools like Postman's network focus more on internal reuse and developer productivity.
How does it discover APIs? Traffic analysis, gateway integration, and code/CI scanning all surface different things. Traffic-based discovery tends to be best at finding shadow APIs that nobody registered anywhere; code-based discovery tends to be better at catching things before they ever go live.
Does it just find APIs, or also act on what it finds? Some tools stop at inventory and risk scoring. Others, like StackHawk, chain discovery directly into testing — which matters if your goal is closing gaps quickly rather than just measuring them.
Multi-gateway reality check. If your organization genuinely runs more than one API gateway (most do), make sure whatever you pick can actually aggregate across all of them, not just the one your team happens to use.

The Bottom Line

API discovery has moved from a "nice to have" governance exercise to something closer to table stakes — driven less by any single trend and more by the combined effect of AI-accelerated development, AI agents as API consumers, and the simple reality that most large organizations have never run a single, unified API stack.

Whichever category of tool fits your organization best, the underlying goal is the same one teams have always had: being able to answer "what APIs do we actually have running right now?" with confidence — not a guess.

Have you evaluated any of these tools, or run into shadow APIs in your own environment? Let us know in the comments — we'll be covering individual platforms in more depth in upcoming posts.