Big Data & Tools with NoSQL
  • Big Data & Tools
  • ReadMe
  • Big Data Overview
    • Overview
    • Job Opportunities
    • What is Data?
    • How does it help?
    • Types of Data
    • The Big 4 V's
      • Variety
      • Volume
      • Velocity
      • Veracity
      • Other V's
    • Trending Technologies
    • Big Data Concerns
    • Big Data Challenges
    • Data Integration
    • Scaling
      • CAP Theorem
      • Optimistic concurrency
      • Eventual consistency
      • Concurrent vs. Parallel Programming
    • Big Data Tools
    • No SQL Databases
    • What does Big Data learning means?
  • Linux & Tools
    • Overview
    • Linux Commands - 01
    • Linux Commands - 02
    • AWK
    • CSVKIT
    • CSVSQL
    • CSVGREP
  • Data Format
    • Storage Formats
    • CSV/TSV/Parquet
    • Parquet Example
    • JSON
    • HTTP & REST API
      • Terms to Know
        • Statefulness
        • Statelessness
        • Monolithic Architecture
        • Microservices
        • Idempotency
    • REST API
    • Python
      • Setup
      • Decorator
      • Unit Testing
      • Flask Demo
      • Flask Demo - 01
      • Flask Demo - 02
      • Flask Demo - 03
      • Flask Demo - 04
      • Flask Demo - 06
    • API Testing
    • Flask Demo Testing
    • API Performance
    • API in Big Data World
  • NoSQL
    • Types of NoSQL Databases
    • Redis
      • Overview
      • Terms to know
      • Redis - (RDBMS) MySql
      • Redis Cache Demo
      • Use Cases
      • Data Structures
        • Strings
        • List
        • Set
        • Hash
        • Geospatial Index
        • Pub/Sub
        • Redis - Python
      • Redis JSON
      • Redis Search
      • Persistence
      • Databases
      • Timeseries
    • Neo4J
      • Introduction
      • Neo4J Terms
      • Software
      • Neo4J Components
      • Hello World
      • Examples
        • MySQL: Neo4J
        • Sample Transactions
        • Sample
        • Create Nodes
        • Update Nodes
        • Relation
        • Putting it all together
        • Commonly used Functions
        • Data Profiling
        • Queries
        • Python Scripts
      • More reading
    • MongoDB
      • Sample JSON
      • Introduction
      • Software
      • MongoDB Best Practices
      • MongoDB Commands
      • Insert Document
      • Querying MongoDB
      • Update & Remove
      • Import
      • Logical Operators
      • Data Types
      • Operators
      • Aggregation Pipeline
      • Further Reading
      • Fun Task
        • Sample
    • InfluxDB
      • Data Format
      • Scripts
  • Python
    • Python Classes
    • Serialization-Deserialization
  • Tools
    • JQ
    • DUCK DB
    • CICD Intro
    • CICD Tools
      • CI YAML
      • CD Yaml
    • Containers
      • VMs or Containers
      • What container does
      • Podman
      • Podman Examples
  • Cloud Everywhere
    • Overview
    • Types of Cloud Services
    • Challenges of Cloud Computing
    • High Availability
    • Azure Cloud
      • Services
      • Storages
      • Demo
    • Terraform
  • Data Engineering
    • Batch vs Streaming
    • Kafka
      • Introduction
      • Kafka Use Cases
      • Kafka Software
      • Python Scripts
      • Different types of Streaming
    • Quality & Governance
    • Medallion Architecture
    • Data Engineering Model
    • Data Mesh
  • Industry Trends
    • Roadmap - Data Engineer
    • Good Reads
      • IP & SUBNET
Powered by GitBook
On this page
  1. NoSQL
  2. InfluxDB

Scripts

mkdir containers
cd containers
mkdir influxdb
cd influxdb

Launch influxdb 1.8 via Docker container

podman run -d --rm -p 8086:8086 \
-e INFLUXDB_DB=db0 \
-e INFLUXDB_ADMIN_ENABLED=true \
-e INFLUXDB_ADMIN_USER=admin \
-e INFLUXDB_ADMIN_PASSWORD=supersecretpassword \
-v "/$PWD/data:/var/lib/influxdb" \
--name=influx_bigdata_class \
influxdb:1.8.10

Launch influx CLI

podman exec -it influx_bigdata_class bash

Invoke SQL Client

influx
create Database homeoffice;

use homeoffice;

INSERT temperature,officename=O1 value=23.1
INSERT temperature,officename=O1 value=23.2
INSERT temperature,officename=O1 value=23.1
INSERT temperature,officename=O1 value=23.3
INSERT temperature,officename=O1 value=23.2

INSERT sound,officename=O1 value=50.1
INSERT sound,officename=O1 value=50.2
INSERT sound,officename=O1 value=55
INSERT sound,officename=O1 value=50.1
INSERT sound,officename=O1 value=50.25
Select Mean("value") from temperature group by "officename"
select * from temperature

Using HTTP API Protocol

curl -i -X POST 'http://localhost:8086/write?db=homeoffice' --data-binary 'temperature,officename=O1 value=99.8'
select * from temperature
curl -G 'http://localhost:8086/query?pretty=true' --data-urlencode "db=homeoffice" --data-urlencode "q=SELECT * FROM temperature WHERE officename='O1'"

Office Database

create database office

use office

show measurements
INSERT temperature,officeName=mainoffice,location=downtown,city=newyork value=72.5

INSERT temperature,officeName=mainoffice,location=downtown,city=newyork value=72.5

INSERT temperature,officeName=branchoffice,location=uptown,city=newyork value=74.2

INSERT temperature,officeName=satelliteoffice,location=financialdistrict,city=sfo value=68.9

INSERT temperature,officeName=regionalhq,location=midtown,city=chicago value=71.1

INSERT temperature,officeName=salesoffice,location=missiondistrict,city=sfo value=69.7

INSERT temperature,officeName=mainoffice,location=downtown,city=newyork value=72.8

INSERT temperature,officeName=branchoffice,location=uptown,city=newyork value=73.9

INSERT temperature,officeName=satelliteoffice,location=financialdistrict,city=sfo value=69.2

INSERT temperature,officeName=regionalhq,location=midtown,city=chicago value=70.5

INSERT temperature,officeName=salesoffice,location=missiondistrict,city=sfo value=70.1


INSERT humidity,officeName=mainoffice,location=downtown,city=newyork value=45.2
INSERT humidity,officeName=branchoffice,location=uptown,city=newyork value=48.7
INSERT humidity,officeName=satelliteoffice,location=financialdistrict,city=sfo value=52.1
INSERT humidity,officeName=regionalhq,location=midtown,city=chicago value=39.8
INSERT humidity,officeName=salesoffice,location=missiondistrict,city=sfo value=51.3
INSERT humidity,officeName=mainoffice,location=downtown,city=newyork value=46.5
INSERT humidity,officeName=branchoffice,location=uptown,city=newyork value=47.9
INSERT humidity,officeName=satelliteoffice,location=financialdistrict,city=sfo value=53.4
INSERT humidity,officeName=regionalhq,location=midtown,city=chicago value=41.2
INSERT humidity,officeName=salesoffice,location=missiondistrict,city=sfo value=50.7

-- Get the maximum temperature and humidity for each office location

SELECT officeName, location, max(value) AS max_temperature FROM temperature GROUP BY officeName, location;

-- Calculate the 3-hour moving average of temperatures for the "Main Office" in New York

SELECT mean("value") AS "moving_avg" FROM "temperature" WHERE "officeName" = 'mainoffice' AND "city" = 'newyork' GROUP BY time(3m)

Quer to compare temperature and humidity for each location

SELECT mean("temp_value") AS "avg_temp", mean("humidity_value") AS "avg_humidity" FROM ( SELECT mean(value) AS "temp_value" FROM "temperature" GROUP BY "location"),(SELECT mean(value) AS "humidity_value" FROM "humidity" GROUP BY "location" ) GROUP BY "location"

PreviousData FormatNextPython Classes

Last updated 1 year ago