Direct Edge MQTT to AWS IoT — Storing Industrial Telemetry in PostgreSQL

In Alberta’s oil and gas industry, data is generated at the edge — not in the cloud. Wellheads, compressors, pipelines, pump jacks, and tank farms continuously produce telemetry such as pressure, temperature, flow rate, vibration, and equipment health metrics.

The real challenge is not collecting this data — it’s moving it securely, reliably, and cost-effectively from the field into systems where engineers can actually use it.

This architecture reflects patterns used in production oil & gas IoT systems across North America.

This blog explains how a Direct Edge MQTT → AWS IoT → Lambda → PostgreSQL architecture solves that problem — and why it’s widely used in modern industrial systems.

This blog illustrates how firmware data flows directly from embedded devices into a production-grade backend database.

🛢️ What Is This Architecture?

Each component has a clear responsibility:

• Edge Device — Collects sensor data and publishes it via MQTT
• AWS IoT Core — Secure device authentication and message routing
• Lambda — Validation, transformation, and business logic
• PostgreSQL (RDS) — Durable storage for analytics, reporting, and dashboards

This is a serverless ingestion pipeline, meaning:

• No backend servers to manage
• Scales automatically as new wells or assets come online — without additional backend engineering effort
• Pay only for usage

❓ Why Not Send Data Directly to the Database?

This is a common question — especially from engineers new to cloud systems.

Direct DB access from edge devices is a bad idea in production:

• ❌ Databases should never be exposed to the public internet
• ❌ Credential management becomes unsafe
• ❌ No validation or filtering
• ❌ No buffering or retry logic
• ❌ Hard to scale and audit

AWS IoT Core acts as a secure gatekeeper between field devices and backend systems.

🔐 Why MQTT + AWS IoT Core?

MQTT is the de-facto protocol for industrial IoT:

• Lightweight (perfect for low-power devices)
• Pub/Sub model scales naturally
• Works over unreliable networks
• Minimal bandwidth usage

AWS IoT Core adds enterprise-grade security:

• X.509 certificate authentication (per device)
• Fine-grained topic-level permissions
• Built-in device registry
• Native integration with AWS services

This is critical in oil & gas environments where:

• Devices are remote
• Networks are intermittent
• Security compliance is mandatory

🧠 Why Store Data in PostgreSQL?

PostgreSQL is a strong fit for industrial telemetry:

• Time-series friendly
• Strong indexing and query performance
• JSONB support for flexible sensor payloads
• Easy integration with dashboards and analytics tools

Typical use cases include:

• Historical trend analysis
• Production reporting
• Alarm investigation
• Regulatory compliance
• Predictive maintenance models

Unlike dashboards alone, databases create long-term operational value.

🛢️ How This Helps Oil & Gas Businesses

Centralized Visibility Field assets are distributed. Decision-making shouldn’t be.

This architecture consolidates telemetry from hundreds or thousands of devices into a single backend — accessible by engineers, analysts, and management.

Reduced Operational Downtime By storing historical data, teams can:

• Detect abnormal trends
• Identify failing equipment early
• Schedule maintenance proactively

This directly reduces:

• Unplanned shutdowns
• Emergency call-outs
• Production losses

Secure by Design Oil & gas environments demand security:

• Device-level authentication
• Encrypted communication
• Private databases inside VPCs
• No direct internet exposure

This architecture meets industrial cybersecurity expectations.

Cost-Efficient Scaling Instead of:

• One server per site
• Dedicated backend infrastructure You get:
• Serverless ingestion
• Automatic scaling
• Usage-based billing

Perfect for pilot projects that later scale to full field deployments.

👷 Why Team Leads Choose This Architecture

This design helps engineering leaders by:

• Reducing firmware complexity at the edge
• Avoiding custom backend services
• Isolating security concerns from device teams
• Allowing teams to scale without re-architecting
• Making ownership boundaries clear (Firmware vs Cloud vs Data)

This means:

• ✔ Faster onboarding of new engineers
• ✔ Fewer production incidents
• ✔ Easier handoff between teams

🔧 How Does the Setup Work?

Step 1 — Edge Device Publishes MQTT Each device publishes telemetry to a structured topic:

devices/{device_id}/telemetry

Example payload:

{
  "device_id": "esp32_001",
  "timestamp": "2026-01-15T10:30:00Z",
  "metrics": {
    "temperature": 72.4,
    "pressure": 101.8,
    "vibration": 0.03
  }
}

Step 2 — AWS IoT Rule Filters Messages An IoT Rule subscribes to:

devices/+/telemetry

It forwards valid messages to AWS Lambda.

This step allows:

• Topic filtering
• Future routing (S3, Kinesis, DynamoDB, etc.)

Step 3 — Lambda Processes the Data Lambda performs:

• Payload validation
• Timestamp normalization
• Schema enforcement
• Database insertion

Because Lambda runs inside a VPC, it securely connects to RDS without exposing the database publicly.

Step 4 — PostgreSQL Stores Telemetry A simple but scalable schema:

CREATE TABLE telemetry (
  id BIGSERIAL PRIMARY KEY,
  device_id VARCHAR(64),
  ts TIMESTAMP,
  metrics JSONB,
  received_at TIMESTAMP DEFAULT NOW()
);

This allows:

• Fast queries by device and time
• Flexible sensor payloads
• Easy integration with dashboards later

Where This Leads Next

• Once data is stored reliably, you unlock:
• Web dashboards (React, Grafana)
• Reporting tools
• Machine learning pipelines
• Predictive maintenance models
• Regulatory reporting

This architecture becomes the foundation — not the end goal.

🤝 Whom Can You Ask for Help?

If you’re building or planning:

• Industrial IoT systems
• Edge-to-cloud pipelines
• Secure device ingestion
• Cloud-backed firmware solutions
• You don’t need multiple vendors.

You need one engineer who understands firmware, cloud, and backend together.

🚀 Quick Call‑to‑Action

Schedule Your Pilot Call

We’ll identify one production line, connect it securely to AWS, and leave you with dashboards ready for leadership.

🚀 Conclusion

Direct Edge MQTT to AWS IoT and PostgreSQL is not a demo architecture — it’s how real industrial systems are built today.

For oil and gas companies, it provides:

• Secure data ingestion
• Operational visibility
• Scalable growth
• Long-term analytics value

If you’re modernizing field operations or building digital oilfield systems, this architecture is a proven starting point.