Posts

Showing posts from December, 2025

Apache NiFi ETL Tutorial for Beginners | Installation & Data Pipeline Basics

Image
In the modern data landscape, the ability to move and transform information seamlessly is what separates a basic system from a professional data architecture. This process is known as ETL (Extract, Transform, Load) . Whether you are a student, an aspiring data engineer, or an IoT enthusiast, mastering ETL tools is a mandatory skill for building automated systems. In this comprehensive guide, I’m going to walk you through Apache NiFi . Unlike many other tools that require heavy coding, NiFi offers a powerful visual interface. I’ve spent a lot of time working with various data orchestrators, and NiFi remains one of my favorites due to its "drag-and-drop" simplicity combined with enterprise-grade power. Today, we will focus on getting it installed, secured, and running in a Windows environment. The Importance of Data Orchestration in IoT In my experience building IoT stations, the biggest challenge isn't just collecting data it's managing the flow. Imagine hav...
Image
What Is MQTT: Complete Guide to MQTT Architecture- Brokers and Cloud Solutions What Is MQTT: Complete Guide to MQTT Architecture, Brokers and Cloud Solutions If you are building an IoT (Internet of Things) project, you quickly realize that standard HTTP requests are too heavy and slow for real-time sensors. This is where MQTT (Message Queuing Telemetry Transport) changes the game. In my experience working with industrial automation and smart meters, MQTT is the gold standard because it is lightweight, keeps bandwidth costs low, and maintains connections even on unstable networks. In this guide, I will move beyond the basic definitions and show you: How the architecture actually works A code example (Python) for sending data The difference between Open-source and Cloud brokers How to install the Mosquitto Broker (Video Tutorial) The Core Concept: Pub/Sub Unlike the web (where you ask a server for a page and it responds), MQTT is...

How to Set Up Apache Kafka on Windows Using KRaft Mode (No ZooKeeper)

Image
How to Install Apache Kafka on Windows (KRaft Mode - No ZooKeeper) Apache Kafka is the industry standard for event streaming. While older versions relied on ZooKeeper, the modern approach is to use KRaft (Kafka Raft Metadata) mode. In this guide, I will walk you through installing Apache Kafka 3.x on Windows without ZooKeeper. This reduces complexity and makes your local setup much lighter. Prerequisite: You must have Java (JDK 11 or JDK 17) installed. Run java -version in your terminal to check before proceeding. Step 1: Download and Extract 1. Go to the official Apache Kafka download page . 2. Download the Binary downloads (Scala 2.13 is recommended). 3. Extract the folder to a short path like D:\kafka . (Avoid long paths with spaces like "Program Files" to prevent errors). Step 2: Navigate to Windows Binaries Open your Command Prompt (CMD) and navig...

ESP32 TFT LCD 240x240 Tutorial-Display Images & DHT11 Sensor

Image
In this comprehensive tutorial, you will learn how to build a real-time temperature and humidity monitoring dashboard using an ESP32 , a 240×240 TFT LCD display (ST7789) , and a DHT11 sensor . The project demonstrates how to display text, icons, and live sensor data on a colorful TFT screen. This guide is designed for makers, students, and engineers who want to improve their skills in embedded systems , IoT development , and ESP32 graphical interfaces . 📺 Video Tutorial – Full Walkthrough This article complements the full step-by-step video tutorial below. In the video, you will see the complete wiring, TFT initialization, bitmap generation, and a live demo of temperature and humidity updates on the screen. 📌 Tip: Watching the video first will make the wiring and code much easier to follow. 🧰 Components Required ESP32 Development Board [Image of ESP32 development board pinout] 240×240 TFT LCD (ST7789 Driver) DHT11 Temperature & Humi...

Import CSV from FTP to PostgreSQL Using Python

Image
Import CSV from FTP to PostgreSQL Using Python | Automated ETL Automating CSV Imports: FTP to PostgreSQL with Python Full ETL Walkthrough & Code Examples In modern data-driven architectures—especially in IoT platforms, smart metering systems, and industrial automation —data rarely originates from a single source. Embedded devices often export measurements as CSV files and upload them to an FTP server because FTP is lightweight and universally supported by legacy hardware. The challenge lies in moving this data into a structured PostgreSQL database for analysis. In this guide, I will show you how to build a Python script that monitors an FTP server, detects new files, and performs an "Upsert" (Update or Insert) into your database. 🛠️ Prerequisites & Architecture Before diving into the code, ensure you have the following Python libraries installed. We use ftplib for the...

How to Set Up Ollama AI in Open Web UI with Docker Desktop

Image
Local AI Revolution: Deploying Ollama & Open Web UI with Docker Running Large Language Models (LLMs) locally offers privacy, zero latency, and freedom from subscription fees. In this guide, we won't just install Ollama ; we will build a full local stack using Docker to connect a powerful backend with the sleek Open Web UI frontend. ⚙️ The Architecture: How It Works Before pasting commands, it is crucial to understand the setup. We are creating two distinct Docker containers that need to communicate: Container A (Backend): Runs Ollama, serving the API on port 11434 . Container B (Frontend): Runs Open Web UI, accessible via your browser on port 3000 . The Bridge: We use the host.docker.internal flag to allow Container B to "see" Container A safely. 📋 Prerequisites Ensure your environment is ready to handle LLM inference: Docker Desktop: Installed and running ( Download...

Control a Relay with ESP32 via Bluetooth Using MIT App Inventor _Part 5

Image
Welcome to the final installment of our series on controlling an ESP32 Relay via Bluetooth. In Part 5, we move beyond basic functionality and focus on App Deployment and UI Customization using MIT App Inventor. A functional app is good, but a usable app is better. In this guide, I will walk you through the specific properties you need to change to brand your app, how to generate the installable APK file, and how to backup your source code using the .aia format. Step 1: UI Customization in Designer View To make your app look professional, you need to modify the Screen1 component properties. Here are the specific settings I use for a clean IoT dashboard: AlignHorizontal: Set to Center (3). This ensures your ON/OFF buttons stay in the middle regardless of phone size. Icon: Upload a 512x512px PNG file here. This replaces the default MIT robot logo on your phone's home screen. Title: Change this from "Screen1" to your project n...

Control a Relay with ESP32 via Bluetooth Using MIT App Inventor _Part 4

Image
In Part 4 of our ESP32 Bluetooth series, we shift focus from the hardware to the software. Today, you will learn how to **build the Android Controller App** from scratch using MIT App Inventor. We will not just drag-and-drop buttons; we will configure the BluetoothClient component to scan for devices and set up the logic to send serial data characters ('1' and '0') that trigger our ESP32 relay. Step 1: Interface Design (Designer View) Before programming the logic, we need to arrange our visual components. Open MIT App Inventor and drag the following components into your workspace: Component Type Name (Rename to) Purpose ListPicker ConnectBtn Opens the list of paired Bluetooth devices (ESP32). Button Btn_ON Sends signal to turn Relay ON. Button Btn_OFF Sends signal to turn Relay OFF. BluetoothClient BluetoothClient1 Non-visible component ...

Control a Relay with ESP32 via Bluetooth Using MIT App Inventor _Part 3

Image
Welcome to Part 3 of our ESP32 Bluetooth Control series. While the code on the ESP32 is important, the User Interface (UI) is what you will interact with every day. In this tutorial, we focus exclusively on the Designer View of MIT App Inventor. We will not just drop buttons randomly; we will use Layout Managers to create a professional, centered dashboard that scales well on different phone screen sizes. Step 1: Setting the Screen Properties Before adding buttons, we need to configure the main screen to handle our components correctly. AlignHorizontal: Change to Center (3) . This ensures all your buttons stay in the middle. AlignVertical: Change to Center (2) for a modern look, or Top (1) if you plan to add a lot of components. Theme: I recommend switching from "Classic" to Device Default for a cleaner, native Android look. Step 2: Using Layout Containers Pro Tip: Never drag buttons directly onto the screen backgroun...

Control a Relay with ESP32 via Bluetooth Using MIT App Inventor _Part 2

Image
Welcome to Part 2 of our Smart Relay series. In Part 1, we handled the physical wiring. Now, we bring the hardware to life by programming the ESP32 . In this tutorial, we will write a C++ sketch using the Arduino IDE that turns the ESP32 into a Bluetooth Classic device, capable of receiving serial data strings from any smartphone. Step 1: Understanding the Hardware Logic Before coding, ensure you know which GPIO pin connects to your relay. In our example, we use GPIO 23 . The ESP32 has built-in Bluetooth and Wi-Fi, making it superior to the older Arduino UNO + HC-05 combination. We will use the Bluetooth Classic protocol (via the BluetoothSerial.h library) because it is simpler to implement for beginner projects than BLE. Step 2: The ESP32 Firmware Copy the following code into your Arduino IDE. This sketch initializes the Bluetooth radio and listens for incoming characters ('1' for ON, '0' for OFF). #include "Bluetoo...

Control a Relay with ESP32 via Bluetooth Using MIT App Inventor

Image
Welcome to Part 1 of our IoT Home Automation series. In this comprehensive guide, we will focus on the Hardware Architecture required to control high-voltage appliances using an ESP32 and a Relay Module via Bluetooth. This isn't just a demo—by the end of this article, you will have a safe, fully wired circuit ready for programming. We will cover the specific voltage requirements of the ESP32 and how to wire the relay terminals (COM/NO/NC) correctly. The Microcontroller: ESP32 We chose the ESP32 over the Arduino UNO for this project because it has built-in Bluetooth Classic and Wi-Fi, eliminating the need for external modules like the HC-05. [Image of ESP32 Pinout diagram] Logic Voltage: 3.3V (Critical for relay selection) Bluetooth: v4.2 BR/EDR and BLE standard GPIOs: We will use GPIO 23 for signaling. The Switching Component: Relay Module A relay is an electromechanical switch. Since the ESP32 operates at 3.3V, ...

Import CSV to PostgreSQL with Python

Image
Import CSV to PostgreSQL with Python: A Simple, Step-by-Step Tutorial for Beginners In the world of data management, CSV files are everywhere from exporting IoT sensor readings to handling employee records or financial logs. But manually copying data from a CSV into PostgreSQL? It's a recipe for frustration, especially as datasets grow. Python to the rescue! This straightforward tutorial shows you how to automate CSV imports using just a few lines of code, making your workflow efficient and error-free. We'll use the built-in csv module and psycopg2 library to read your CSV file and insert rows directly into a PostgreSQL table. No fancy frameworks like Pandas needed for this basic setup keeping it lightweight for beginners. By the end, you'll have a reusable script that handles sample data like staff records (ID, name, age) and scales to real IoT exports. Whether you're tracking device metrics or building a simple database app, th...

How to Install InfluxDB V3 and Create DB, Write & Query Explained

Image
InfluxDB V3 represents a massive architectural shift in time-series databases. By rebuilding the core engine around the Apache Arrow ecosystem (using DataFusion and Parquet), V3 offers massive data ingestion rates and, most importantly, native SQL querying support. In this guide, we will move beyond the theory and walk through the practical steps: installing the CLI, creating your first database, writing data using Line Protocol, and querying it back using standard SQL. Why InfluxDB V3 is Different The biggest change in V3 is the separation of storage and compute, known as the IOx engine . Unlike V2 which used the Flux language, V3 embraces SQL. Ingest: Still uses the lightweight Line Protocol (perfect for IoT). Query: Now uses standard SQL (easier for analysts). Storage: Persists data as Parquet files (highly compressed). Step 1: Installing the CLI Environment To interact with InfluxDB V3 (whether running locally ...

How to Backup and Restore PostgreSQL Databases with pgAdmin

Image
If you are working with PostgreSQL databases, relying solely on command-line tools like pg_dump can be intimidating. Fortunately, pgAdmin 4 provides a robust Graphical User Interface (GUI) to handle backups seamlessly. In this guide, we will cover not just the basic clicks, but the critical configuration settings (like Binary Paths and Format types) that usually cause errors for beginners. Critical Prerequisite: Binary Path Configuration Before you start: The most common error users face is "Utility file not found." This happens because pgAdmin doesn't know where your PostgreSQL installation is. Go to File > Preferences . Navigate to Paths > Binary paths . Scroll to "PostgreSQL Binary Path". Set it to your bin folder (e.g., C:\Program Files\PostgreSQL\16\bin ). Step 1: Creating a Backup The backup process creates a snapshot of your schema and data. Right-click your database name in the browser...

Contact Form

Name

Email *

Message *