Big Data & Tools with NoSQL
  • Big Data & Tools
  • ReadMe
  • Big Data Overview
    • Overview
    • Job Opportunities
    • What is Data?
    • How does it help?
    • Types of Data
    • The Big 4 V's
      • Variety
      • Volume
      • Velocity
      • Veracity
      • Other V's
    • Trending Technologies
    • Big Data Concerns
    • Big Data Challenges
    • Data Integration
    • Scaling
      • CAP Theorem
      • Optimistic concurrency
      • Eventual consistency
      • Concurrent vs. Parallel Programming
    • Big Data Tools
    • No SQL Databases
    • What does Big Data learning means?
  • Linux & Tools
    • Overview
    • Linux Commands - 01
    • Linux Commands - 02
    • AWK
    • CSVKIT
    • CSVSQL
    • CSVGREP
  • Data Format
    • Storage Formats
    • CSV/TSV/Parquet
    • Parquet Example
    • JSON
    • HTTP & REST API
      • Terms to Know
        • Statefulness
        • Statelessness
        • Monolithic Architecture
        • Microservices
        • Idempotency
    • REST API
    • Python
      • Setup
      • Decorator
      • Unit Testing
      • Flask Demo
      • Flask Demo - 01
      • Flask Demo - 02
      • Flask Demo - 03
      • Flask Demo - 04
      • Flask Demo - 06
    • API Testing
    • Flask Demo Testing
    • API Performance
    • API in Big Data World
  • NoSQL
    • Types of NoSQL Databases
    • Redis
      • Overview
      • Terms to know
      • Redis - (RDBMS) MySql
      • Redis Cache Demo
      • Use Cases
      • Data Structures
        • Strings
        • List
        • Set
        • Hash
        • Geospatial Index
        • Pub/Sub
        • Redis - Python
      • Redis JSON
      • Redis Search
      • Persistence
      • Databases
      • Timeseries
    • Neo4J
      • Introduction
      • Neo4J Terms
      • Software
      • Neo4J Components
      • Hello World
      • Examples
        • MySQL: Neo4J
        • Sample Transactions
        • Sample
        • Create Nodes
        • Update Nodes
        • Relation
        • Putting it all together
        • Commonly used Functions
        • Data Profiling
        • Queries
        • Python Scripts
      • More reading
    • MongoDB
      • Sample JSON
      • Introduction
      • Software
      • MongoDB Best Practices
      • MongoDB Commands
      • Insert Document
      • Querying MongoDB
      • Update & Remove
      • Import
      • Logical Operators
      • Data Types
      • Operators
      • Aggregation Pipeline
      • Further Reading
      • Fun Task
        • Sample
    • InfluxDB
      • Data Format
      • Scripts
  • Python
    • Python Classes
    • Serialization-Deserialization
  • Tools
    • JQ
    • DUCK DB
    • CICD Intro
    • CICD Tools
      • CI YAML
      • CD Yaml
    • Containers
      • VMs or Containers
      • What container does
      • Podman
      • Podman Examples
  • Cloud Everywhere
    • Overview
    • Types of Cloud Services
    • Challenges of Cloud Computing
    • High Availability
    • Azure Cloud
      • Services
      • Storages
      • Demo
    • Terraform
  • Data Engineering
    • Batch vs Streaming
    • Kafka
      • Introduction
      • Kafka Use Cases
      • Kafka Software
      • Python Scripts
      • Different types of Streaming
    • Quality & Governance
    • Medallion Architecture
    • Data Engineering Model
    • Data Mesh
  • Industry Trends
    • Roadmap - Data Engineer
    • Good Reads
      • IP & SUBNET
Powered by GitBook
On this page
  • Properties
  • Basic Rules
  • JSON Values
  • Sample JSON Document
  • JSON Best Practices
  • Use Cases:
  • Serialize
  1. Data Format

JSON

Java Script Object Notation.

This is neither a row-based nor Columnar Format.

The flexible way to store & share data across systems.

It's a text file with curly braces & key-value pairs { }

Simplest JSON format

{"id": "1","name":"Rachel"}

Properties

  • Language Independent.

  • Self-describing and easy to understand.

Basic Rules

  • Curly braces to hold the objects.

  • Data is represented in Key Value or Name Value pairs.

  • Data is separated by a comma.

  • The use of double quotes is necessary.

  • Square brackets [ ] hold an array of data.

JSON Values

String  {"name":"Rachel"}

Number  {"id":101}

Boolean {"result":true, "status":false}  (lowercase)

Object  {
            "character":{"fname":"Rachel","lname":"Green"}
        }

Array   {
            "characters":["Rachel","Ross","Joey","Chanlder"]
        }

NULL    {"id":null}

Sample JSON Document

{
    "characters": [
        {
            "id" : 1,
            "fName":"Rachel",
            "lName":"Green",
            "status":true
        },
        {
            "id" : 2,
            "fName":"Ross",
            "lName":"Geller",
            "status":true
        },
        {
            "id" : 3,
            "fName":"Chandler",
            "lName":"Bing",
            "status":true
        },
        {
            "id" : 4,
            "fName":"Phebe",
            "lName":"Buffay",
            "status":false
        }
    ]
}

JSON Best Practices

No Hyphen in your Keys.

{"first-name":"Rachel","last-name":"Green"}  is not right. ✘

Under Scores Okay

{"first_name":"Rachel","last_name":"Green"} is okay ✓

Lowercase Okay

{"firstname":"Rachel","lastname":"Green"} is okay ✓

Camelcase best

{"firstName":"Rachel","lastName":"Green"} is the best. ✓

Use Cases:

  • APIs and Web Services: JSON is widely used in RESTful APIs for sending and receiving data.

  • Configuration Files: Many modern applications and development tools use JSON for configuration.

  • Data Storage: Some NoSQL databases like MongoDB use JSON or BSON (binary JSON) for storing data.

  • Serialization and Deserialization: Converting data to/from a format that can be stored or transmitted.

Serialize

Convert Python Object to JSON Format

import json

# Python dictionary with Friend's characters
friends_characters = {
    "characters": [
        {"name": "Rachel Green", "job": "Fashion Executive"},
        {"name": "Ross Geller", "job": "Paleontologist"},
        {"name": "Monica Geller", "job": "Chef"},
        {"name": "Chandler Bing", "job": "Statistical Analysis and Data Reconfiguration"},
        {"name": "Joey Tribbiani", "job": "Actor"},
        {"name": "Phoebe Buffay", "job": "Massage Therapist"}
    ]
}

print(type(friends_characters), friends_characters)

print("-" * 200 )

# Serializing json
json_data = json.dumps(friends_characters, indent=4)
print(type(json_data), json_data)

# Saving to a file
with open('friends_characters.json', 'w') as file:
    json.dump(friends_characters, file, indent=4)

DeSerialize

Convert JSON Format/File to Python Object

import json

# JSON string with Friend's characters
json_data = '''
{
    "characters": [
        {"name": "Rachel Green", "job": "Fashion Executive"},
        {"name": "Ross Geller", "job": "Paleontologist"},
        {"name": "Monica Geller", "job": "Chef"},
        {"name": "Chandler Bing", "job": "Statistical Analysis and Data Reconfiguration"},
        {"name": "Joey Tribbiani", "job": "Actor"},
        {"name": "Phoebe Buffay", "job": "Massage Therapist"}
    ]
}
'''

# Parsing JSON to dictionary
friends_characters = json.loads(json_data)
print(friends_characters, type(friends_characters))

## Alternate way to read from file

# Path to your JSON file
file_path = 'friends_characters.json'

# Open the file and read the JSON content
with open(file_path, 'r') as file:
    data = json.load(file)
print(data, type(data))
PreviousParquet ExampleNextHTTP & REST API

Last updated 1 year ago