Articles by Shiv Iyer

About Shiv Iyer

Open Source Database Systems Engineer with a deep understanding of Optimizer Internals, Performance Engineering, Scalability and Data SRE. Shiv currently is the Founder, Investor, Board Member and CEO of multiple Database Systems Infrastructure Operations companies in the Transaction Processing Computing and ColumnStores ecosystem. He is also a frequent speaker in open source software conferences globally.

How to tune Redis for High Traffic Applications?

Redis Optimization Guide for High-Traffic Applications Tuning Redis for high-traffic applications requires careful optimization of configuration, hardware, data structures, and monitoring. Here's a comprehensive guide to ensure Redis handles high traffic efficiently: 1. Redis Optimization [...]

Preventing PostgreSQL Transaction ID Wraparound

Preventing PostgreSQL Transaction ID Wraparound: Managing Autovacuum Blockages and Ensuring Database Health PostgreSQL Transaction ID Wraparound Transaction ID (XID) wraparound is a critical concern in PostgreSQL databases, as it can render a database unusable by [...]

Milvus Architecture and Internals

Milvus Architecture Milvus is a purpose-built vector database optimized for similarity search and AI workloads. Its architecture is distinct from traditional RDBMS like PostgreSQL and MySQL, as it is designed to handle high-dimensional vector data [...]

Comprehensive Guide to Implementing Data Encryption in MongoDB for Secure Data Management

Secure Data Management with MongoDB Data Encryption MongoDB's comprehensive data encryption framework encompasses three critical security layers: data at rest encryption, secure communication channels, and advanced key management protocols. This guide outlines the essential implementation [...]

Scalable Write Optimization Strategies for Milvus

Scalable Write Optimization Strategies for Milvus: Best Practices and Techniques Optimizing write operations in Milvus, an advanced vector database system, is essential for maintaining high performance in applications that require real-time data ingestion, large-scale data [...]

Understanding Lost Updates in PostgreSQL

Understanding and Preventing PostgreSQL Lost Updates Executive Summary PostgreSQL Lost updates present a significant challenge in database management, particularly when handling concurrent transactions in PostgreSQL. This comprehensive guide examines the technical aspects of lost updates [...]

PL/pgSQL Application to Log Slow Queries in PostgreSQL Server

Creating a PL/pgSQL Application to Log Slow Queries in PostgreSQL Server Creating a PostgreSQL application using PL/pgSQL to record all queries executed on the PostgreSQL server with a latency/response time of more than 1 second [...]

Using pgBadger for Continuous PostgreSQL Monitoring and Performance Optimization

Comprehensive Guide to Using pgBadger for Continuous PostgreSQL Monitoring and Performance Optimization Implementation of pgBadger for continuous PostgreSQL monitoring involves a systematic approach encompassing log configuration, analysis, and report generation. This advanced tool conducts a [...]

Difference Between WAL Files and Transaction Logs in PostgreSQL

Understanding the Difference Between WAL Files and Transaction Logs in PostgreSQL In PostgreSQL, two key components that are frequently discussed in relation to database management and performance are the transaction log and the WAL (Write-Ahead [...]

Implementing a Tamper-Proof Ledger in PostgreSQL

Implementing a Tamper-Proof Ledger in PostgreSQL for Auditable and Secure Data Storage In PostgreSQL, a ledger refers to a specialized database structure or extension designed to provide immutable, auditable, and secure data storage. This structure [...]

The Data Transformation Company

Data Architecture, Engineering and Operations for SQL, NoSQL, NewSQL, Cloud Native Data Platforms, Analytics and AI